[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:31 dist-test-slave-srgv krb5kdc[19107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:31 dist-test-slave-srgv krb5kdc[19107](info): set up 2 sockets
Aug 12 05:15:31 dist-test-slave-srgv krb5kdc[19107](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 05:15:32.250454 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:32 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:32.390470 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35733
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:32.486332 19120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:32.486828 19120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:32.486924 19120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:32.489100 19120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:32.489245 19120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:32.489271 19120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:32.489295 19120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:32.489323 19120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:32.491787 19120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:32.492282 19120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:32.492558 19120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:32.500641 19126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:32.500711 19120 server_base.cc:1034] running on GCE node
W20260812 05:15:32.501749 19128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:32.502080 19125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:32.502487 19120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:32.502964 19120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:32.504130 19120 hybrid_clock.cc:648] HybridClock initialized: now 1786511732504112 us; error 68 us; skew 500 ppm
Aug 12 05:15:32 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511732, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:32.508000 19120 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:32.508705 19120 webserver.cc:460] Webserver started at http://127.18.166.254:41735/ using document root <none> and password file <none>
I20260812 05:15:32.508992 19120 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:32.509047 19120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:32.509199 19120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:32.510749 19120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "baf9937771e5424c8f51ffed136b0c2f"
format_stamp: "Formatted at 2026-08-12 05:15:32 on dist-test-slave-srgv"
server_key: "d8f82006c4eb74bb7c5fe69bc0c92081"
server_key_iv: "5bb3b89fa15b873e42ff98e6bd34f31e"
server_key_version: "encryptionkey@0"
I20260812 05:15:32.511199 19120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "baf9937771e5424c8f51ffed136b0c2f"
format_stamp: "Formatted at 2026-08-12 05:15:32 on dist-test-slave-srgv"
server_key: "d8f82006c4eb74bb7c5fe69bc0c92081"
server_key_iv: "5bb3b89fa15b873e42ff98e6bd34f31e"
server_key_version: "encryptionkey@0"
I20260812 05:15:32.513697 19120 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 05:15:32.515080 19135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:32.515446 19120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:32.515543 19120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "baf9937771e5424c8f51ffed136b0c2f"
format_stamp: "Formatted at 2026-08-12 05:15:32 on dist-test-slave-srgv"
server_key: "d8f82006c4eb74bb7c5fe69bc0c92081"
server_key_iv: "5bb3b89fa15b873e42ff98e6bd34f31e"
server_key_version: "encryptionkey@0"
I20260812 05:15:32.515631 19120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:32.549914 19120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:32.553442 19120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:32.553658 19120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:32.566507 19120 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:35733
I20260812 05:15:32.566752 19187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:35733 every 8 connection(s)
I20260812 05:15:32.567418 19120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:32.569244 19188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:32.573716 19188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f: Bootstrap starting.
I20260812 05:15:32.575081 19188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:32.575994 19188 log.cc:824] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:32.577126 19188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f: No bootstrap required, opened a new log
I20260812 05:15:32.577781 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19120
I20260812 05:15:32.577876 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:32.578539 19099 external_mini_cluster.cc:1428] Setting key f2d20a2ceec15e915675ccb1eae30aab
I20260812 05:15:32.581756 19188 raft_consensus.cc:348] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf9937771e5424c8f51ffed136b0c2f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35733 } }
I20260812 05:15:32.582010 19188 raft_consensus.cc:374] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:32.582187 19188 raft_consensus.cc:729] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf9937771e5424c8f51ffed136b0c2f, State: Initialized, Role: FOLLOWER
I20260812 05:15:32.582406 19188 consensus_queue.cc:260] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf9937771e5424c8f51ffed136b0c2f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35733 } }
I20260812 05:15:32.582568 19188 raft_consensus.cc:388] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:32.582620 19188 raft_consensus.cc:482] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:32.582675 19188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:32.584012 19188 raft_consensus.cc:504] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf9937771e5424c8f51ffed136b0c2f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35733 } }
I20260812 05:15:32.584288 19188 leader_election.cc:302] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baf9937771e5424c8f51ffed136b0c2f; no voters:
I20260812 05:15:32.584597 19188 leader_election.cc:288] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:32.584681 19193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:32.584971 19193 raft_consensus.cc:686] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [term 1 LEADER]: Becoming Leader. State: Replica: baf9937771e5424c8f51ffed136b0c2f, State: Running, Role: LEADER
I20260812 05:15:32.585150 19193 consensus_queue.cc:237] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf9937771e5424c8f51ffed136b0c2f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35733 } }
I20260812 05:15:32.585223 19188 sys_catalog.cc:564] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:32.585872 19193 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader baf9937771e5424c8f51ffed136b0c2f. Latest consensus state: current_term: 1 leader_uuid: "baf9937771e5424c8f51ffed136b0c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf9937771e5424c8f51ffed136b0c2f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35733 } } }
I20260812 05:15:32.585979 19193 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:32.586452 19195 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baf9937771e5424c8f51ffed136b0c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf9937771e5424c8f51ffed136b0c2f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35733 } } }
I20260812 05:15:32.586601 19195 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:32.587494 19202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:32.588961 19202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:32.592422 19202 catalog_manager.cc:1347] Generated new cluster ID: dfbde11cbbe24538a8e079cbdfc6f37b
I20260812 05:15:32.592543 19202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:15:32 dist-test-slave-srgv krb5kdc[19107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:32.603487 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:32.579944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.18.166.254:35733)
0812 05:15:32.580451 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:32.580464 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:32.580735 (+ 271us) server_negotiation.cc:407] Connection header received
0812 05:15:32.583233 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:32.583239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:32.583335 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:32.583501 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:32.586129 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:32.588154 (+ 2025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:32.589979 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:32.590502 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:32.590661 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:32.597736 (+ 7075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:32.597743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:32.597744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:32.597762 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:32.600392 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:32.601287 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:32.601291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:32.601292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:32.601498 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:32.601896 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:32.601897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:32.601897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:32.602250 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:32.602417 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:32.602947 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 05:15:32.603188 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":270,"thread_start_us":119,"threads_started":1}
I20260812 05:15:32.619310 19202 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:32.620043 19202 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:32.630352 19202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P baf9937771e5424c8f51ffed136b0c2f: Generated new TSK 0
I20260812 05:15:32.630715 19202 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:32.668427 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35733
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:32.779847 19216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:32.780171 19216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:32.780232 19216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:32.781744 19216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:32.781836 19216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:32.781893 19216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:32.783816 19216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35733
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:32.784276 19216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:32.784574 19216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:32.793800 19224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:32.793833 19221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:32.793833 19222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:32.794478 19216 server_base.cc:1034] running on GCE node
I20260812 05:15:32.794759 19216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:32.795076 19216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:32.796252 19216 hybrid_clock.cc:648] HybridClock initialized: now 1786511732796207 us; error 66 us; skew 500 ppm
Aug 12 05:15:32 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511732, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:32.800599 19216 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:32.801272 19216 webserver.cc:460] Webserver started at http://127.18.166.193:43145/ using document root <none> and password file <none>
I20260812 05:15:32.801609 19216 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:32.801692 19216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:32.801877 19216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:32.803249 19216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "7a4e753285c74b39a6f7471c630c50ac"
format_stamp: "Formatted at 2026-08-12 05:15:32 on dist-test-slave-srgv"
server_key: "b371e5ba4d6dcd00c0ee953be7325a9e"
server_key_iv: "fa8a871bf746e84bb6dbfbdf2d84a072"
server_key_version: "encryptionkey@0"
I20260812 05:15:32.803726 19216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "7a4e753285c74b39a6f7471c630c50ac"
format_stamp: "Formatted at 2026-08-12 05:15:32 on dist-test-slave-srgv"
server_key: "b371e5ba4d6dcd00c0ee953be7325a9e"
server_key_iv: "fa8a871bf746e84bb6dbfbdf2d84a072"
server_key_version: "encryptionkey@0"
I20260812 05:15:32.805511 19216 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:15:32.806445 19233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:32.806703 19216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:32.806831 19216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "7a4e753285c74b39a6f7471c630c50ac"
format_stamp: "Formatted at 2026-08-12 05:15:32 on dist-test-slave-srgv"
server_key: "b371e5ba4d6dcd00c0ee953be7325a9e"
server_key_iv: "fa8a871bf746e84bb6dbfbdf2d84a072"
server_key_version: "encryptionkey@0"
I20260812 05:15:32.806917 19216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:32.835080 19216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:32.837322 19216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:32.837612 19216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:32.838007 19216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:32.838446 19216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:32.838544 19216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:32.838593 19216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:32.838624 19216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:32.851984 19216 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:42539
I20260812 05:15:32.852108 19355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:42539 every 8 connection(s)
I20260812 05:15:32.852483 19216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:32.856941 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19216
I20260812 05:15:32.857036 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:32.857198 19099 external_mini_cluster.cc:1428] Setting key 995bcf906747e72aeac4bf11cd1870b4
Aug 12 05:15:32 dist-test-slave-srgv krb5kdc[19107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511732, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:32.865689 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:32.853569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:40647 (local address 127.18.166.254:35733)
0812 05:15:32.853666 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:32.853668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:32.854037 (+ 369us) server_negotiation.cc:407] Connection header received
0812 05:15:32.854989 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:32.854990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:32.855005 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:32.855064 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:32.856458 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:32.857045 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:32.858506 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:32.858763 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:32.858966 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:32.862101 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:32.862104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:32.862104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:32.862113 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:32.863662 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:32.864349 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:32.864350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:32.864350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:32.864413 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:32.864771 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:32.864771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:32.864772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:32.864894 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:32.864945 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:32.865495 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 05:15:32.865606 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 05:15:32.866012 19358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:32.853520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35733 (local address 127.18.166.193:40647)
0812 05:15:32.853860 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:32.853903 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:32.854881 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:32.855149 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:32.855153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:32.855972 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:32.856344 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:32.856351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:32.857185 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:32.857187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:32.858398 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:32.858402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:32.859005 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:32.859006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:32.859213 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:32.859977 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:32.859986 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:32.861966 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:32.863786 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:32.863799 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:32.863803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:32.864246 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:32.864489 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:32.864490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:32.864491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:32.864691 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:32.864964 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:32.864974 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:32.865177 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 05:15:32.865344 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 05:15:32.865689 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":187,"threads_started":1}
I20260812 05:15:32.866806 19356 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35733
I20260812 05:15:32.866963 19356 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:32.867218 19356 heartbeater.cc:511] Master 127.18.166.254:35733 requested a full tablet report, sending...
I20260812 05:15:32.868237 19152 ts_manager.cc:194] Registered new tserver with Master: 7a4e753285c74b39a6f7471c630c50ac (127.18.166.193:42539)
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:15:32.869099 19152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:40647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:32.889663 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35733
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:33.002919 19361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:33.003250 19361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:33.003300 19361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:33.004789 19361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:33.004894 19361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:33.004947 19361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:33.007010 19361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35733
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:33.007381 19361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:33.007670 19361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:33.014824 19369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:33.014932 19367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:33.014986 19361 server_base.cc:1034] running on GCE node
W20260812 05:15:33.015081 19366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:33.015398 19361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:33.015720 19361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:33.016916 19361 hybrid_clock.cc:648] HybridClock initialized: now 1786511733016886 us; error 41 us; skew 500 ppm
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:33.020946 19361 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:33.021570 19361 webserver.cc:460] Webserver started at http://127.18.166.194:35233/ using document root <none> and password file <none>
I20260812 05:15:33.021848 19361 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:33.021900 19361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:33.022081 19361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:33.023321 19361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "939d7cb352f24610a23e7e3fc2eb56e2"
format_stamp: "Formatted at 2026-08-12 05:15:33 on dist-test-slave-srgv"
server_key: "2dee1a6b820edc6cde9ef8faabde4398"
server_key_iv: "8bbc2e07e966cef93d879fc53fcee7bf"
server_key_version: "encryptionkey@0"
I20260812 05:15:33.023808 19361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "939d7cb352f24610a23e7e3fc2eb56e2"
format_stamp: "Formatted at 2026-08-12 05:15:33 on dist-test-slave-srgv"
server_key: "2dee1a6b820edc6cde9ef8faabde4398"
server_key_iv: "8bbc2e07e966cef93d879fc53fcee7bf"
server_key_version: "encryptionkey@0"
I20260812 05:15:33.025790 19361 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:15:33.026746 19376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:33.026969 19361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:33.027071 19361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "939d7cb352f24610a23e7e3fc2eb56e2"
format_stamp: "Formatted at 2026-08-12 05:15:33 on dist-test-slave-srgv"
server_key: "2dee1a6b820edc6cde9ef8faabde4398"
server_key_iv: "8bbc2e07e966cef93d879fc53fcee7bf"
server_key_version: "encryptionkey@0"
I20260812 05:15:33.027161 19361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:33.058276 19361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:33.060904 19361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:33.061094 19361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:33.061429 19361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:33.061870 19361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:33.061941 19361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:33.061978 19361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:33.062008 19361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:33.076671 19361 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:39731
I20260812 05:15:33.076792 19494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:39731 every 8 connection(s)
I20260812 05:15:33.077262 19361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:33.086949 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19361
I20260812 05:15:33.087071 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:33.087280 19099 external_mini_cluster.cc:1428] Setting key 07c43041a824f646f4b4d2d081f469b2
I20260812 05:15:33.089958 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:33.078238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:58269 (local address 127.18.166.254:35733)
0812 05:15:33.078317 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:33.078319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:33.078669 (+ 350us) server_negotiation.cc:407] Connection header received
0812 05:15:33.079424 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:33.079426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:33.079439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:33.079499 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:33.080731 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.081247 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.082507 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.082715 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.082774 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:33.085854 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:33.085856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:33.085857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:33.085866 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:33.088073 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.088550 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.088551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.088552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.088624 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.088886 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.088887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.088887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.089048 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:33.089125 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:33.089728 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 05:15:33.089834 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:15:33.089958 19497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:33.078210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35733 (local address 127.18.166.194:58269)
0812 05:15:33.078511 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:33.078554 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:33.079302 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:33.079561 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:33.079565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:33.080297 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:33.080624 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:33.080630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.081360 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.081360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:33.082417 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:33.082421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.082803 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.082804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:33.082973 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:33.083670 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:33.083679 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:33.085734 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:33.088186 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:33.088195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:33.088198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:33.088467 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:33.088705 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:33.088705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:33.088705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:33.088818 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:33.089115 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:33.089120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:33.089254 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 05:15:33.089361 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 05:15:33.089609 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":183,"threads_started":1}
I20260812 05:15:33.091034 19495 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35733
I20260812 05:15:33.091272 19495 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:33.091542 19495 heartbeater.cc:511] Master 127.18.166.254:35733 requested a full tablet report, sending...
I20260812 05:15:33.092039 19152 ts_manager.cc:194] Registered new tserver with Master: 939d7cb352f24610a23e7e3fc2eb56e2 (127.18.166.194:39731)
I20260812 05:15:33.092633 19152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:58269
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:33.121367 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35733
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:33.243152 19500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:33.243454 19500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:33.243484 19500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:33.245074 19500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:33.245148 19500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:33.245183 19500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:33.247301 19500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35733
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:33.247646 19500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:33.247938 19500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:33.254724 19506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:33.254683 19508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:33.254918 19500 server_base.cc:1034] running on GCE node
W20260812 05:15:33.254706 19505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:33.255303 19500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:33.255609 19500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:33.256817 19500 hybrid_clock.cc:648] HybridClock initialized: now 1786511733256811 us; error 42 us; skew 500 ppm
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:33.260169 19500 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:33.260805 19500 webserver.cc:460] Webserver started at http://127.18.166.195:42983/ using document root <none> and password file <none>
I20260812 05:15:33.261076 19500 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:33.261152 19500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:33.261322 19500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:33.262699 19500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "9382c4e050354288a10c762e621ec4b9"
format_stamp: "Formatted at 2026-08-12 05:15:33 on dist-test-slave-srgv"
server_key: "05fb559447e8015f4dc9540cf32654c4"
server_key_iv: "80e09f2552cf2eced20cba58ae677ad1"
server_key_version: "encryptionkey@0"
I20260812 05:15:33.263159 19500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "9382c4e050354288a10c762e621ec4b9"
format_stamp: "Formatted at 2026-08-12 05:15:33 on dist-test-slave-srgv"
server_key: "05fb559447e8015f4dc9540cf32654c4"
server_key_iv: "80e09f2552cf2eced20cba58ae677ad1"
server_key_version: "encryptionkey@0"
I20260812 05:15:33.265082 19500 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 05:15:33.266260 19515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:33.266510 19500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:33.266637 19500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "9382c4e050354288a10c762e621ec4b9"
format_stamp: "Formatted at 2026-08-12 05:15:33 on dist-test-slave-srgv"
server_key: "05fb559447e8015f4dc9540cf32654c4"
server_key_iv: "80e09f2552cf2eced20cba58ae677ad1"
server_key_version: "encryptionkey@0"
I20260812 05:15:33.266736 19500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:33.306906 19500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:33.309973 19500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:33.310168 19500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:33.310518 19500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:33.310909 19500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:33.310947 19500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:33.310993 19500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:33.311019 19500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:33.327105 19500 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:35213
I20260812 05:15:33.327283 19635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:35213 every 8 connection(s)
I20260812 05:15:33.327638 19500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:33.328996 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19500
I20260812 05:15:33.329130 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:33.329362 19099 external_mini_cluster.cc:1428] Setting key 2fd17fbe6dc22b7567e37e26d90c7eee
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:33.339632 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:33.328517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:53819 (local address 127.18.166.254:35733)
0812 05:15:33.328610 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:33.328613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:33.329143 (+ 530us) server_negotiation.cc:407] Connection header received
0812 05:15:33.330105 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:33.330106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:33.330120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:33.330189 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:33.331731 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.332241 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.333521 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.333731 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.333817 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:33.336467 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:33.336468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:33.336469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:33.336479 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:33.338062 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.338546 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.338548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.338549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.338621 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.338910 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.338911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.338911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.339040 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:33.339125 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:33.339430 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 05:15:33.339521 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:15:33.339869 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:33.328501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35733 (local address 127.18.166.195:53819)
0812 05:15:33.328923 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:33.328985 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:33.329990 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:33.330283 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:33.330289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:33.331266 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:33.331612 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:33.331618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.332352 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.332353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:33.333373 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:33.333376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.333829 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.333830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:33.333993 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:33.334654 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:33.334664 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:33.336336 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:33.338177 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:33.338189 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:33.338192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:33.338459 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:33.338704 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:33.338705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:33.338705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:33.338834 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:33.339114 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:33.339123 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:33.339284 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 05:15:33.339406 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 05:15:33.339589 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":217,"threads_started":1}
I20260812 05:15:33.340457 19636 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35733
I20260812 05:15:33.340575 19636 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:33.340744 19636 heartbeater.cc:511] Master 127.18.166.254:35733 requested a full tablet report, sending...
I20260812 05:15:33.341187 19152 ts_manager.cc:194] Registered new tserver with Master: 9382c4e050354288a10c762e621ec4b9 (127.18.166.195:35213)
I20260812 05:15:33.341775 19152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:53819
I20260812 05:15:33.341862 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:33.369030 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:33.360241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.18.166.193:42539)
0812 05:15:33.360458 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:33.360463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:33.360478 (+ 15us) server_negotiation.cc:407] Connection header received
0812 05:15:33.360506 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:33.360508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:33.360522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:33.360605 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:33.361264 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.362072 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.362829 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.363172 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.363233 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:33.365856 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:33.365876 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:33.365876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:33.365886 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:33.367695 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.368116 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.368125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.368125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.368186 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.368406 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.368407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.368407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.368568 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:33.368742 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:33.368788 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 05:15:33.368884 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":128,"threads_started":1}
W20260812 05:15:33.369674 19262 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36644
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:33.395346 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:33.387778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.18.166.193:42539)
0812 05:15:33.387853 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:33.387855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:33.387932 (+ 77us) server_negotiation.cc:407] Connection header received
0812 05:15:33.387986 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:33.387987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:33.387997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:33.388050 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:33.388658 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.389137 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.389931 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:33.390139 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:33.390201 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:33.392705 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:33.392706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:33.392706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:33.392714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:33.394236 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.394614 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.394615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.394615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.394673 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:33.394855 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:33.394856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:33.394856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:33.394957 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:33.395036 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:33.395154 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:15:33.395229 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:15:33.396021 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19216
I20260812 05:15:33.403785 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19361
I20260812 05:15:33.410861 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19500
I20260812 05:15:33.416913 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19120
2026-08-12T05:15:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2254 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19650](info): set up 2 sockets
Aug 12 05:15:33 dist-test-slave-srgv krb5kdc[19650](info): commencing operation
krb5kdc: starting...
W20260812 05:15:34.500737 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:34 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:34.637596 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42825
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:34.724700 19663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:34.724970 19663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:34.724996 19663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:34.726364 19663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:34.726431 19663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:34.726446 19663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:34.726459 19663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:34.726471 19663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:34.728178 19663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:42825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:34.728489 19663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:34.728780 19663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:15:34.734395 19663 server_base.cc:1034] running on GCE node
W20260812 05:15:34.734407 19669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:34.734407 19668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:34.734392 19671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:34.734930 19663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:34.735301 19663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:34.736493 19663 hybrid_clock.cc:648] HybridClock initialized: now 1786511734736476 us; error 36 us; skew 500 ppm
Aug 12 05:15:34 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:34.739535 19663 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:34.740108 19663 webserver.cc:460] Webserver started at http://127.18.166.254:34375/ using document root <none> and password file <none>
I20260812 05:15:34.740365 19663 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:34.740437 19663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:34.740599 19663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:34.741858 19663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1"
format_stamp: "Formatted at 2026-08-12 05:15:34 on dist-test-slave-srgv"
server_key: "3307c6d58fc5ba3cbb1fd97afd670e77"
server_key_iv: "71cf7a76ac26c9a564636391c31342dc"
server_key_version: "encryptionkey@0"
I20260812 05:15:34.742257 19663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1"
format_stamp: "Formatted at 2026-08-12 05:15:34 on dist-test-slave-srgv"
server_key: "3307c6d58fc5ba3cbb1fd97afd670e77"
server_key_iv: "71cf7a76ac26c9a564636391c31342dc"
server_key_version: "encryptionkey@0"
I20260812 05:15:34.743848 19663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:34.744861 19680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:34.745136 19663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:34.745261 19663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1"
format_stamp: "Formatted at 2026-08-12 05:15:34 on dist-test-slave-srgv"
server_key: "3307c6d58fc5ba3cbb1fd97afd670e77"
server_key_iv: "71cf7a76ac26c9a564636391c31342dc"
server_key_version: "encryptionkey@0"
I20260812 05:15:34.745358 19663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:34.762998 19663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:34.765738 19663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:34.765949 19663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:34.770778 19663 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:42825
I20260812 05:15:34.770843 19733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:42825 every 8 connection(s)
I20260812 05:15:34.771288 19663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:34.772338 19734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:34.773729 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19663
I20260812 05:15:34.773833 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:34.773999 19099 external_mini_cluster.cc:1428] Setting key 192decffa5ef90169135f350d74d245d
I20260812 05:15:34.775693 19734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1: Bootstrap starting.
I20260812 05:15:34.776499 19734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:34.776875 19734 log.cc:824] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:34.777834 19734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1: No bootstrap required, opened a new log
I20260812 05:15:34.780550 19734 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42825 } }
I20260812 05:15:34.780725 19734 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:34.780748 19734 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8df7752c0674fdfa46d77b9ea2a4aa1, State: Initialized, Role: FOLLOWER
I20260812 05:15:34.780893 19734 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42825 } }
I20260812 05:15:34.780977 19734 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:34.781014 19734 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:34.781075 19734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:34.781899 19734 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42825 } }
I20260812 05:15:34.782047 19734 leader_election.cc:302] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8df7752c0674fdfa46d77b9ea2a4aa1; no voters:
Aug 12 05:15:34 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:34.782287 19734 leader_election.cc:288] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:34.782486 19739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:34.782627 19734 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:34.782743 19739 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [term 1 LEADER]: Becoming Leader. State: Replica: b8df7752c0674fdfa46d77b9ea2a4aa1, State: Running, Role: LEADER
I20260812 05:15:34.782858 19739 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42825 } }
I20260812 05:15:34.783275 19739 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8df7752c0674fdfa46d77b9ea2a4aa1. Latest consensus state: current_term: 1 leader_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42825 } } }
I20260812 05:15:34.783414 19739 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:34.783288 19740 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8df7752c0674fdfa46d77b9ea2a4aa1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42825 } } }
I20260812 05:15:34.783687 19740 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:34.783876 19748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:34.784809 19748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:34.786854 19748 catalog_manager.cc:1347] Generated new cluster ID: 06097b193b644b47be240781737b9614
I20260812 05:15:34.786912 19748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:34.787515 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:34.774661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.18.166.254:42825)
0812 05:15:34.774912 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:34.774923 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:34.774964 (+ 41us) server_negotiation.cc:407] Connection header received
0812 05:15:34.775557 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:34.775579 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:34.775648 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:34.775831 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:34.776576 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:34.778004 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:34.778836 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:34.779520 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:34.779673 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:34.782764 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:34.782769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:34.782769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:34.782785 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:34.785395 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:34.785910 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:34.785913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:34.785913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:34.786037 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:34.786283 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:34.786284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:34.786285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:34.786584 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:34.786761 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:34.787159 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 05:15:34.787306 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":113,"threads_started":1}
I20260812 05:15:34.811556 19748 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:34.812192 19748 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:34.823697 19748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8df7752c0674fdfa46d77b9ea2a4aa1: Generated new TSK 0
I20260812 05:15:34.823947 19748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:34.869294 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42825
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:34.962752 19760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:34.962996 19760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:34.963024 19760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:34.964452 19760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:34.964534 19760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:34.964567 19760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:34.966493 19760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:34.966828 19760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:34.967110 19760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:34.972568 19766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:34.972572 19765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:34.972839 19760 server_base.cc:1034] running on GCE node
W20260812 05:15:34.972568 19768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:34.973196 19760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:34.973574 19760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:34.974783 19760 hybrid_clock.cc:648] HybridClock initialized: now 1786511734974762 us; error 49 us; skew 500 ppm
Aug 12 05:15:34 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:34.978096 19760 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:34.978750 19760 webserver.cc:460] Webserver started at http://127.18.166.193:39189/ using document root <none> and password file <none>
I20260812 05:15:34.979022 19760 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:34.979122 19760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:34.979295 19760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:34.980469 19760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "e70f0f5b977c424a8e829a7922733780"
format_stamp: "Formatted at 2026-08-12 05:15:34 on dist-test-slave-srgv"
server_key: "51967c01b69f3f4e6cc237ad3a657126"
server_key_iv: "ca000c258ce907257aa3c97845b72267"
server_key_version: "encryptionkey@0"
I20260812 05:15:34.980883 19760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "e70f0f5b977c424a8e829a7922733780"
format_stamp: "Formatted at 2026-08-12 05:15:34 on dist-test-slave-srgv"
server_key: "51967c01b69f3f4e6cc237ad3a657126"
server_key_iv: "ca000c258ce907257aa3c97845b72267"
server_key_version: "encryptionkey@0"
I20260812 05:15:34.982573 19760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:15:34.983704 19775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:34.984077 19760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:34.984186 19760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "e70f0f5b977c424a8e829a7922733780"
format_stamp: "Formatted at 2026-08-12 05:15:34 on dist-test-slave-srgv"
server_key: "51967c01b69f3f4e6cc237ad3a657126"
server_key_iv: "ca000c258ce907257aa3c97845b72267"
server_key_version: "encryptionkey@0"
I20260812 05:15:34.984287 19760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:35.032321 19760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:35.035400 19760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:35.035640 19760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:35.035950 19760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:35.036324 19760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:35.036389 19760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.036437 19760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:35.036465 19760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.043627 19760 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:46277
I20260812 05:15:35.043659 19893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:46277 every 8 connection(s)
I20260812 05:15:35.044183 19760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:35.047099 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19760
I20260812 05:15:35.047224 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:35.047482 19099 external_mini_cluster.cc:1428] Setting key 7bbc562b9cb5156446e81d87104f5b0c
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:15:35.058689 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.045149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:39181 (local address 127.18.166.254:42825)
0812 05:15:35.045202 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.045204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.045607 (+ 403us) server_negotiation.cc:407] Connection header received
0812 05:15:35.046906 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.046909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.046925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.046994 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.048144 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.048984 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.050837 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.051245 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.051364 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.054176 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.054179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.054179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.054189 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.056262 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.056812 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.056813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.056814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.056889 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.057263 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.057265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.057265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.057476 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.057562 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.058199 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.058409 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:15:35.059010 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.045120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42825 (local address 127.18.166.193:39181)
0812 05:15:35.045369 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.045470 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.046747 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.047088 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.047093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.047585 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:35.048030 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.048039 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.049160 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.049162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.050695 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.050700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.051395 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.051396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.051533 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.052260 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:35.052269 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:35.054037 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:35.056390 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:35.056401 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:35.056405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:35.056692 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:35.056995 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:35.056996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:35.056996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:35.057143 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:35.057551 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:35.057559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:35.057722 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.058115 (+ 393us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.058623 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":119,"threads_started":1}
I20260812 05:15:35.059842 19894 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42825
I20260812 05:15:35.059957 19894 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:35.060278 19894 heartbeater.cc:511] Master 127.18.166.254:42825 requested a full tablet report, sending...
I20260812 05:15:35.061036 19698 ts_manager.cc:194] Registered new tserver with Master: e70f0f5b977c424a8e829a7922733780 (127.18.166.193:46277)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
I20260812 05:15:35.061866 19698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:39181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:35.080806 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42825
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:35.168252 19900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:35.168503 19900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:35.168533 19900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:35.169966 19900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:35.170039 19900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:35.170080 19900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:35.171941 19900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:35.172286 19900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:35.172562 19900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:35.178092 19905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:35.178014 19906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:35.178000 19908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:35.178527 19900 server_base.cc:1034] running on GCE node
I20260812 05:15:35.178774 19900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:35.179008 19900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:35.180202 19900 hybrid_clock.cc:648] HybridClock initialized: now 1786511735180190 us; error 30 us; skew 500 ppm
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:35.183212 19900 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:35.183815 19900 webserver.cc:460] Webserver started at http://127.18.166.194:44101/ using document root <none> and password file <none>
I20260812 05:15:35.184141 19900 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:35.184196 19900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:35.184301 19900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:35.185478 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "cf986e2ddb5b45c3a5039abe03d2c11b"
format_stamp: "Formatted at 2026-08-12 05:15:35 on dist-test-slave-srgv"
server_key: "cae4ac3d24a4bcc51997884f044aa79f"
server_key_iv: "b1136f20599543bed19b81ba0b1ec651"
server_key_version: "encryptionkey@0"
I20260812 05:15:35.185832 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "cf986e2ddb5b45c3a5039abe03d2c11b"
format_stamp: "Formatted at 2026-08-12 05:15:35 on dist-test-slave-srgv"
server_key: "cae4ac3d24a4bcc51997884f044aa79f"
server_key_iv: "b1136f20599543bed19b81ba0b1ec651"
server_key_version: "encryptionkey@0"
I20260812 05:15:35.187255 19900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:35.188138 19915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.188346 19900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:35.188428 19900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "cf986e2ddb5b45c3a5039abe03d2c11b"
format_stamp: "Formatted at 2026-08-12 05:15:35 on dist-test-slave-srgv"
server_key: "cae4ac3d24a4bcc51997884f044aa79f"
server_key_iv: "b1136f20599543bed19b81ba0b1ec651"
server_key_version: "encryptionkey@0"
I20260812 05:15:35.188493 19900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:35.204352 19900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:35.207170 19900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:35.207317 19900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:35.207554 19900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:35.207870 19900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:35.207903 19900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.207927 19900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:35.207940 19900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.214560 19900 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:42431
I20260812 05:15:35.214613 20038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:42431 every 8 connection(s)
I20260812 05:15:35.214982 19900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:35.217176 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 19900
I20260812 05:15:35.217283 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:35.217496 19099 external_mini_cluster.cc:1428] Setting key e0ce86170e8e96ef33bda2652e608db5
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:15:35.226470 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.215820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:32943 (local address 127.18.166.254:42825)
0812 05:15:35.215896 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.215898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.216152 (+ 254us) server_negotiation.cc:407] Connection header received
0812 05:15:35.217307 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.217309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.217326 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.217398 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.218150 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.218671 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.220213 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.220545 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.220625 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.223327 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.223328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.223329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.223339 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.224885 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.225372 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.225374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.225374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.225466 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.225750 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.225751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.225751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.225883 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.225960 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.226271 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.226367 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:15:35.226688 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.215801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42825 (local address 127.18.166.194:32943)
0812 05:15:35.215974 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.216016 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.217149 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.217503 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.217508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.217872 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:35.218076 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.218081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.218815 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.218816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.220118 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.220121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.220645 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.220646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.220762 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.221367 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:35.221379 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:35.223212 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:35.224994 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:35.225005 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:35.225008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:35.225286 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:35.225535 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:35.225536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:35.225537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:35.225668 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:35.225961 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:35.225970 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:35.226124 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.226249 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.226437 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260812 05:15:35.227267 20039 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42825
I20260812 05:15:35.227396 20039 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:35.227655 20039 heartbeater.cc:511] Master 127.18.166.254:42825 requested a full tablet report, sending...
I20260812 05:15:35.228121 19698 ts_manager.cc:194] Registered new tserver with Master: cf986e2ddb5b45c3a5039abe03d2c11b (127.18.166.194:42431)
I20260812 05:15:35.228539 19698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:32943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:35.246459 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42825
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:35.332947 20045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:35.333196 20045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:35.333226 20045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:35.334579 20045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:35.334640 20045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:35.334673 20045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:35.336285 20045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:35.336514 20045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:35.336731 20045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:35.342341 20053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:35.342362 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:35.342341 20051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:35.342602 20045 server_base.cc:1034] running on GCE node
I20260812 05:15:35.342849 20045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:35.343181 20045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:35.344352 20045 hybrid_clock.cc:648] HybridClock initialized: now 1786511735344337 us; error 33 us; skew 500 ppm
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:35.347239 20045 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:35.347805 20045 webserver.cc:460] Webserver started at http://127.18.166.195:40503/ using document root <none> and password file <none>
I20260812 05:15:35.348084 20045 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:35.348148 20045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:35.348345 20045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:35.349629 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "b8bc4cf117784992b8e0cf5df4a42055"
format_stamp: "Formatted at 2026-08-12 05:15:35 on dist-test-slave-srgv"
server_key: "b6ef3c2765d4dfd60ccb1f399c6aa792"
server_key_iv: "2b7a59be9c3fcd5e3bbee5105c2e736c"
server_key_version: "encryptionkey@0"
I20260812 05:15:35.350049 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "b8bc4cf117784992b8e0cf5df4a42055"
format_stamp: "Formatted at 2026-08-12 05:15:35 on dist-test-slave-srgv"
server_key: "b6ef3c2765d4dfd60ccb1f399c6aa792"
server_key_iv: "2b7a59be9c3fcd5e3bbee5105c2e736c"
server_key_version: "encryptionkey@0"
I20260812 05:15:35.351599 20045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:15:35.352515 20060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.352835 20045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:35.352979 20045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "b8bc4cf117784992b8e0cf5df4a42055"
format_stamp: "Formatted at 2026-08-12 05:15:35 on dist-test-slave-srgv"
server_key: "b6ef3c2765d4dfd60ccb1f399c6aa792"
server_key_iv: "2b7a59be9c3fcd5e3bbee5105c2e736c"
server_key_version: "encryptionkey@0"
I20260812 05:15:35.353070 20045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:35.376637 20045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:35.379086 20045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:35.379271 20045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:35.379551 20045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:35.379917 20045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:35.379953 20045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.379977 20045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:35.380029 20045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:35.386771 20045 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:36895
I20260812 05:15:35.386898 20178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:36895 every 8 connection(s)
I20260812 05:15:35.387293 20045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:35.392632 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 20045
I20260812 05:15:35.392750 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:35.392951 19099 external_mini_cluster.cc:1428] Setting key 9cc5160d4ffef5fc26e13513b6408db8
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:35.397917 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.388254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:51465 (local address 127.18.166.254:42825)
0812 05:15:35.388302 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.388304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.388526 (+ 222us) server_negotiation.cc:407] Connection header received
0812 05:15:35.389568 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.389570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.389581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.389663 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.390321 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.390842 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.391954 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.392153 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.392215 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.394912 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.394914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.394914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.394921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.396489 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.396898 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.396899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.396899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.396956 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.397179 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.397179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.397180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.397295 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.397349 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.397745 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.397828 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:15:35.398056 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.388209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42825 (local address 127.18.166.195:51465)
0812 05:15:35.388386 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.388432 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.389385 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.389732 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.389736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.390024 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:35.390232 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.390236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.390938 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.390938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.391871 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.391874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.392241 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.392241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.392349 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.393053 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:35.393064 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:35.394823 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:35.396574 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:35.396580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:35.396583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:35.396830 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:35.397015 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:35.397016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:35.397016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:35.397122 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:35.397354 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:35.397358 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:35.397568 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.397676 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.397829 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260812 05:15:35.398756 20179 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42825
I20260812 05:15:35.398854 20179 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:35.399104 20179 heartbeater.cc:511] Master 127.18.166.254:42825 requested a full tablet report, sending...
I20260812 05:15:35.399519 19698 ts_manager.cc:194] Registered new tserver with Master: b8bc4cf117784992b8e0cf5df4a42055 (127.18.166.195:36895)
I20260812 05:15:35.399875 19698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:51465
I20260812 05:15:35.404994 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:15:35.413777 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.407126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.18.166.254:42825)
0812 05:15:35.407218 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.407220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.407328 (+ 108us) server_negotiation.cc:407] Connection header received
0812 05:15:35.407434 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.407435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.407448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.407520 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.408134 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.408716 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.409502 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.409710 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.409791 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.411091 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.411093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.411094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.411102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.412383 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.412790 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.412791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.412792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.412859 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.413083 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.413084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.413085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.413265 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.413383 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.413550 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.413657 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:15:35.417158 19698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:15:35.419247 19698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:35.426851 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.422983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:36895 (local address 127.0.0.1:48548)
0812 05:15:35.423362 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.423373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.423425 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.423702 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.423703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.423706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:35.423751 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.423753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.424779 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.424780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.425623 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.425626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.426493 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.426495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.426584 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.426600 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.426693 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.426760 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":64,"threads_started":1}
I20260812 05:15:35.426859 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.423184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.18.166.195:36895)
0812 05:15:35.423556 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.423559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.423568 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:35.423598 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.423600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.423611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.423673 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:35.423844 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.424646 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.425721 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.426396 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.426497 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.426562 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.426727 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.426778 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":84,"threads_started":1}
I20260812 05:15:35.427178 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.422682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44768 (local address 127.18.166.193:46277)
0812 05:15:35.422882 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.422887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.423027 (+ 140us) server_negotiation.cc:407] Connection header received
0812 05:15:35.423163 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.423165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.423181 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.423233 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:35.423498 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.424437 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.425969 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.426721 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.426773 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.426818 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.427044 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.427089 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":119,"threads_started":1}
I20260812 05:15:35.427441 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.422552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:46277 (local address 127.0.0.1:44768)
0812 05:15:35.422906 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.422955 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.423054 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.423355 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.423356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.423357 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:35.423407 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.423409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.424547 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.424549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.425871 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.425873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.426881 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.426882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.426943 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.426948 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.427310 (+ 362us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.427366 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260812 05:15:35.428308 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.422774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:42431 (local address 127.0.0.1:37954)
0812 05:15:35.422931 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.422955 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.423040 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.423320 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.423324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.423326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:35.423407 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.423409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.425831 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.425832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.426922 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.426923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.428046 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.428047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.428113 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.428117 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.428171 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.428225 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260812 05:15:35.428377 19825 tablet_service.cc:1467] Processing CreateTablet for tablet 56de3decb2e444399bec235736be40c6 (DEFAULT_TABLE table=test-table [id=ca5bc516a4cc4d629c0a56002ebf8245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:35.428654 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.422910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37954 (local address 127.18.166.194:42431)
0812 05:15:35.423116 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.423121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.423132 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:15:35.423188 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.423189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.423206 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.423302 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:35.424589 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.425705 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.427027 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.427944 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.428432 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.428502 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.428530 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.428579 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260812 05:15:35.428812 19825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56de3decb2e444399bec235736be40c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:35.429212 20109 tablet_service.cc:1467] Processing CreateTablet for tablet 56de3decb2e444399bec235736be40c6 (DEFAULT_TABLE table=test-table [id=ca5bc516a4cc4d629c0a56002ebf8245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:35.429569 20109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56de3decb2e444399bec235736be40c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:35.431917 20196 tablet_bootstrap.cc:492] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: Bootstrap starting.
I20260812 05:15:35.432461 20197 tablet_bootstrap.cc:492] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: Bootstrap starting.
I20260812 05:15:35.432590 20196 tablet_bootstrap.cc:654] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:35.433113 20197 tablet_bootstrap.cc:654] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:35.433386 20196 log.cc:824] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:35.433753 20197 log.cc:824] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:35.433758 19966 tablet_service.cc:1467] Processing CreateTablet for tablet 56de3decb2e444399bec235736be40c6 (DEFAULT_TABLE table=test-table [id=ca5bc516a4cc4d629c0a56002ebf8245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:35.434147 19966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56de3decb2e444399bec235736be40c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:35.434378 20196 tablet_bootstrap.cc:492] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: No bootstrap required, opened a new log
I20260812 05:15:35.434461 20196 ts_tablet_manager.cc:1397] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:35.434556 20197 tablet_bootstrap.cc:492] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: No bootstrap required, opened a new log
I20260812 05:15:35.434623 20197 ts_tablet_manager.cc:1397] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:15:35.436615 20197 raft_consensus.cc:348] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.436686 20196 raft_consensus.cc:348] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.436847 20196 raft_consensus.cc:374] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:35.436847 20197 raft_consensus.cc:374] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:35.436887 20196 raft_consensus.cc:729] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70f0f5b977c424a8e829a7922733780, State: Initialized, Role: FOLLOWER
I20260812 05:15:35.436887 20197 raft_consensus.cc:729] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8bc4cf117784992b8e0cf5df4a42055, State: Initialized, Role: FOLLOWER
I20260812 05:15:35.437043 20197 consensus_queue.cc:260] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.437043 20196 consensus_queue.cc:260] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.437378 20197 ts_tablet_manager.cc:1428] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:15:35.437378 20196 ts_tablet_manager.cc:1428] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:15:35.437474 20200 tablet_bootstrap.cc:492] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: Bootstrap starting.
I20260812 05:15:35.437539 19894 heartbeater.cc:503] Master 127.18.166.254:42825 was elected leader, sending a full tablet report...
I20260812 05:15:35.437749 20179 heartbeater.cc:503] Master 127.18.166.254:42825 was elected leader, sending a full tablet report...
I20260812 05:15:35.438306 20200 tablet_bootstrap.cc:654] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:35.439150 20200 log.cc:824] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:35.440099 20200 tablet_bootstrap.cc:492] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: No bootstrap required, opened a new log
I20260812 05:15:35.440188 20200 ts_tablet_manager.cc:1397] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:35.441816 20200 raft_consensus.cc:348] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.442006 20200 raft_consensus.cc:374] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:35.442055 20200 raft_consensus.cc:729] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf986e2ddb5b45c3a5039abe03d2c11b, State: Initialized, Role: FOLLOWER
I20260812 05:15:35.442214 20200 consensus_queue.cc:260] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.442548 20200 ts_tablet_manager.cc:1428] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:15:35.442591 20039 heartbeater.cc:503] Master 127.18.166.254:42825 was elected leader, sending a full tablet report...
W20260812 05:15:35.465644 20040 tablet.cc:2369] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:15:35.545063 19896 tablet.cc:2369] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:15:35.637975 20180 tablet.cc:2369] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:15:35.722215 20201 raft_consensus.cc:482] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:35.722455 20201 raft_consensus.cc:504] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.723080 20201 leader_election.cc:288] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e70f0f5b977c424a8e829a7922733780 (127.18.166.193:46277), cf986e2ddb5b45c3a5039abe03d2c11b (127.18.166.194:42431)
I20260812 05:15:35.726344 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.723439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:49889 (local address 127.18.166.193:46277)
0812 05:15:35.723531 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.723534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.723547 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:15:35.723583 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.723584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.723594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.723692 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:35.723913 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.724471 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.725551 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.725956 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.726016 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.726024 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.726217 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.726264 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 05:15:35.726343 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.723344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:46277 (local address 127.18.166.195:49889)
0812 05:15:35.723408 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.723422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.723472 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.723717 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.723718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.723720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:35.723816 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.723818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.724567 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.724568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.725460 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.725463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.726046 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.726048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.726139 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.726143 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.726201 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.726239 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":30}
I20260812 05:15:35.726456 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.723613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:59353 (local address 127.18.166.194:42431)
0812 05:15:35.723703 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.723705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.723753 (+ 48us) server_negotiation.cc:407] Connection header received
0812 05:15:35.723820 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.723821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.723831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.723889 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:35.724072 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.724561 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.725633 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.726043 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.726120 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.726128 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.726330 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.726381 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 05:15:35.726688 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.723488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:42431 (local address 127.18.166.195:59353)
0812 05:15:35.723692 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:35.723704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:35.723772 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:35.723911 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:35.723912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:35.723914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:35.723987 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.723988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.724657 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.724658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.725532 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:35.725535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.726137 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.726138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:35.726239 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.726244 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:15:35.726303 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:15:35.726341 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":116,"threads_started":1}
I20260812 05:15:35.726876 19846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56de3decb2e444399bec235736be40c6" candidate_uuid: "b8bc4cf117784992b8e0cf5df4a42055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70f0f5b977c424a8e829a7922733780" is_pre_election: true
I20260812 05:15:35.727070 19846 raft_consensus.cc:2454] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8bc4cf117784992b8e0cf5df4a42055 in term 0.
I20260812 05:15:35.727378 19989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56de3decb2e444399bec235736be40c6" candidate_uuid: "b8bc4cf117784992b8e0cf5df4a42055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" is_pre_election: true
I20260812 05:15:35.727398 20063 leader_election.cc:302] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8bc4cf117784992b8e0cf5df4a42055, e70f0f5b977c424a8e829a7922733780; no voters:
I20260812 05:15:35.727538 19989 raft_consensus.cc:2454] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8bc4cf117784992b8e0cf5df4a42055 in term 0.
I20260812 05:15:35.727555 20201 raft_consensus.cc:2781] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:35.727607 20201 raft_consensus.cc:482] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:35.727636 20201 raft_consensus.cc:3037] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:35.728438 20201 raft_consensus.cc:504] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.728583 20201 leader_election.cc:288] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [CANDIDATE]: Term 1 election: Requested vote from peers e70f0f5b977c424a8e829a7922733780 (127.18.166.193:46277), cf986e2ddb5b45c3a5039abe03d2c11b (127.18.166.194:42431)
I20260812 05:15:35.728806 19989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56de3decb2e444399bec235736be40c6" candidate_uuid: "b8bc4cf117784992b8e0cf5df4a42055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b"
I20260812 05:15:35.728912 19989 raft_consensus.cc:3037] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:35.729521 19846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56de3decb2e444399bec235736be40c6" candidate_uuid: "b8bc4cf117784992b8e0cf5df4a42055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70f0f5b977c424a8e829a7922733780"
I20260812 05:15:35.729607 19846 raft_consensus.cc:3037] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:35.729934 19989 raft_consensus.cc:2454] T 56de3decb2e444399bec235736be40c6 P cf986e2ddb5b45c3a5039abe03d2c11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8bc4cf117784992b8e0cf5df4a42055 in term 1.
I20260812 05:15:35.730165 20063 leader_election.cc:302] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8bc4cf117784992b8e0cf5df4a42055, cf986e2ddb5b45c3a5039abe03d2c11b; no voters:
I20260812 05:15:35.730320 20201 raft_consensus.cc:2781] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:35.730535 20201 raft_consensus.cc:686] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [term 1 LEADER]: Becoming Leader. State: Replica: b8bc4cf117784992b8e0cf5df4a42055, State: Running, Role: LEADER
I20260812 05:15:35.730621 19846 raft_consensus.cc:2454] T 56de3decb2e444399bec235736be40c6 P e70f0f5b977c424a8e829a7922733780 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8bc4cf117784992b8e0cf5df4a42055 in term 1.
I20260812 05:15:35.730665 20201 consensus_queue.cc:237] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } }
I20260812 05:15:35.731696 19698 catalog_manager.cc:5495] T 56de3decb2e444399bec235736be40c6 P b8bc4cf117784992b8e0cf5df4a42055 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8bc4cf117784992b8e0cf5df4a42055 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "b8bc4cf117784992b8e0cf5df4a42055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70f0f5b977c424a8e829a7922733780" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 46277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf986e2ddb5b45c3a5039abe03d2c11b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 42431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8bc4cf117784992b8e0cf5df4a42055" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36895 } health_report { overall_health: HEALTHY } } }
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:35.787319 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.778589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.18.166.193:46277)
0812 05:15:35.778682 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.778685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.778771 (+ 86us) server_negotiation.cc:407] Connection header received
0812 05:15:35.778885 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.778886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.778898 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.778973 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.779629 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.780143 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.780922 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.781126 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.781206 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.783935 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.783960 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.783960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.783969 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.785964 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.786393 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.786397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.786397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.786467 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.786681 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.786682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.786682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.786843 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.787004 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.787094 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.787188 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:35.814371 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.806633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.18.166.193:46277)
0812 05:15:35.806691 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.806693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.806766 (+ 73us) server_negotiation.cc:407] Connection header received
0812 05:15:35.806818 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.806819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.806829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.806884 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.807568 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.808069 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.808810 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.809016 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.809076 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.811766 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.811768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.811768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.811777 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.813202 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.813632 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.813633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.813633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.813701 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.813876 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.813876 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.813876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.813976 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.814046 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.814200 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.814271 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[19650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:35.839504 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:35.831662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.18.166.193:46277)
0812 05:15:35.831744 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:35.831746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:35.831796 (+ 50us) server_negotiation.cc:407] Connection header received
0812 05:15:35.831859 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:35.831860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:35.831871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:35.831926 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:35.832550 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.833045 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.833902 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:35.834109 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:35.834182 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:35.836701 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:35.836703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:35.836703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:35.836712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:35.838247 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.838620 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.838622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.838622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.838690 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:35.838910 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:35.838912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:35.838912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:35.839088 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:35.839210 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:35.839343 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 05:15:35.839427 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 05:15:35.840353 19805 tablet_service.cc:2908] Rejecting scan request for tablet 56de3decb2e444399bec235736be40c6: Uninitialized: clean time has not yet been initialized
I20260812 05:15:35.841625 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19760
I20260812 05:15:35.849556 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19900
I20260812 05:15:35.856987 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 20045
I20260812 05:15:35.863251 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 19663
2026-08-12T05:15:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2446 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[20221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[20221](info): set up 2 sockets
Aug 12 05:15:35 dist-test-slave-srgv krb5kdc[20221](info): commencing operation
krb5kdc: starting...
W20260812 05:15:36.963150 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:37.093268 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39129
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:39129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:37.178726 20235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:37.179014 20235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:37.179071 20235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:37.180361 20235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:37.180447 20235 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 05:15:37.180470 20235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:37.180492 20235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:37.180526 20235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:37.180555 20235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:37.182545 20235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:39129
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:37.182905 20235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:37.183182 20235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:15:37.188540 20235 server_base.cc:1034] running on GCE node
W20260812 05:15:37.188493 20240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.188509 20243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.188513 20241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:37.189200 20235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:37.189628 20235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:37.190788 20235 hybrid_clock.cc:648] HybridClock initialized: now 1786511737190776 us; error 34 us; skew 500 ppm
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:37.193706 20235 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:37.194260 20235 webserver.cc:460] Webserver started at http://127.18.166.254:33547/ using document root <none> and password file <none>
I20260812 05:15:37.194492 20235 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:37.194537 20235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:37.194650 20235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:37.195767 20235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "0c8b44205799488a945024be570e2922"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "feb12e523394d46d115b6013bbec70f7"
server_key_iv: "a8462b4d5290e6751ee0a2eaa0d706a6"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.196125 20235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "0c8b44205799488a945024be570e2922"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "feb12e523394d46d115b6013bbec70f7"
server_key_iv: "a8462b4d5290e6751ee0a2eaa0d706a6"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.197685 20235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 05:15:37.198498 20250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.198710 20235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:37.198802 20235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "0c8b44205799488a945024be570e2922"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "feb12e523394d46d115b6013bbec70f7"
server_key_iv: "a8462b4d5290e6751ee0a2eaa0d706a6"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.198868 20235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:37.210793 20235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:37.213119 20235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:37.213259 20235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:37.218875 20235 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:39129
I20260812 05:15:37.219022 20316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:39129 every 8 connection(s)
I20260812 05:15:37.219357 20235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:37.220155 20317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:37.222805 20317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: Bootstrap starting.
I20260812 05:15:37.223484 20317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:37.223817 20317 log.cc:824] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:37.224653 20317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: No bootstrap required, opened a new log
I20260812 05:15:37.227133 20317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b44205799488a945024be570e2922" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39129 } }
I20260812 05:15:37.227329 20317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:37.227352 20317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8b44205799488a945024be570e2922, State: Initialized, Role: FOLLOWER
I20260812 05:15:37.227535 20317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b44205799488a945024be570e2922" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39129 } }
I20260812 05:15:37.227627 20317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:37.227676 20317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:37.227734 20317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:37.228461 20317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b44205799488a945024be570e2922" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39129 } }
I20260812 05:15:37.228596 20317 leader_election.cc:302] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8b44205799488a945024be570e2922; no voters:
I20260812 05:15:37.228853 20317 leader_election.cc:288] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:37.228945 20322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:37.229131 20322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [term 1 LEADER]: Becoming Leader. State: Replica: 0c8b44205799488a945024be570e2922, State: Running, Role: LEADER
I20260812 05:15:37.229135 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 20235
I20260812 05:15:37.229229 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:37.229197 20317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:37.229445 19099 external_mini_cluster.cc:1428] Setting key d49b047819befe473b714a3991c65add
I20260812 05:15:37.229401 20322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b44205799488a945024be570e2922" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39129 } }
I20260812 05:15:37.229866 20323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c8b44205799488a945024be570e2922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b44205799488a945024be570e2922" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39129 } } }
I20260812 05:15:37.229954 20323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:37.230204 20324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8b44205799488a945024be570e2922. Latest consensus state: current_term: 1 leader_uuid: "0c8b44205799488a945024be570e2922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b44205799488a945024be570e2922" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39129 } } }
I20260812 05:15:37.230607 20324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:37.230508 20332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:37.231294 20332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:37.233064 20332 catalog_manager.cc:1347] Generated new cluster ID: 9818c2a66d0843d093994e1a68acfb73
I20260812 05:15:37.233129 20332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:37.241182 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.230172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.18.166.254:39129)
0812 05:15:37.230659 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.230670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.230725 (+ 55us) server_negotiation.cc:407] Connection header received
0812 05:15:37.230896 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.230905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.231054 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.231207 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.231936 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.233244 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.234119 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.234597 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.234731 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.237322 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.237328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.237328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.237345 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.239371 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.239824 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.239828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.239828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.239959 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.240209 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.240210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.240210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.240511 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.240728 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.240803 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.240969 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":79,"threads_started":1}
I20260812 05:15:37.252456 20332 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:37.253050 20332 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:37.258842 20332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: Generated new TSK 0
I20260812 05:15:37.259074 20332 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:37.293147 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39129
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:37.380296 20347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:37.380589 20347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:37.380635 20347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:37.381979 20347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:37.382054 20347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:37.382102 20347 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:15:37.382160 20347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:37.383755 20347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:39129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:37.384078 20347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:37.384341 20347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:37.390118 20353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.390134 20356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.390118 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:37.390365 20347 server_base.cc:1034] running on GCE node
I20260812 05:15:37.390686 20347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:37.390980 20347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:37.392143 20347 hybrid_clock.cc:648] HybridClock initialized: now 1786511737392156 us; error 60 us; skew 500 ppm
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:37.395273 20347 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:37.395955 20347 webserver.cc:460] Webserver started at http://127.18.166.193:44919/ using document root <none> and password file <none>
I20260812 05:15:37.396186 20347 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:37.396234 20347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:37.396339 20347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:37.397496 20347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "45fac252c0ff43698257866ecd17decb"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "8ae9b643ae2b74bfc3a1aea3c336ab1f"
server_key_iv: "12b595e3dfd9ea05fe65a87b7c38e87c"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.397878 20347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "45fac252c0ff43698257866ecd17decb"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "8ae9b643ae2b74bfc3a1aea3c336ab1f"
server_key_iv: "12b595e3dfd9ea05fe65a87b7c38e87c"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.399299 20347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:15:37.400136 20364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.400413 20347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:15:37.400520 20347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "45fac252c0ff43698257866ecd17decb"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "8ae9b643ae2b74bfc3a1aea3c336ab1f"
server_key_iv: "12b595e3dfd9ea05fe65a87b7c38e87c"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.400611 20347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:37.411134 20347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:37.413360 20347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:37.413627 20347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:37.414202 20347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:37.414240 20347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.414265 20347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:37.414315 20347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:37.422438 20347 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:43701
I20260812 05:15:37.422549 20483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:43701 every 8 connection(s)
I20260812 05:15:37.422938 20347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:37.425675 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.414915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:43027 (local address 127.18.166.254:39129)
0812 05:15:37.414998 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.415001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.415300 (+ 299us) server_negotiation.cc:407] Connection header received
0812 05:15:37.416111 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.416113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.416128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.416232 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.417018 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.417757 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.418861 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.419102 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.419224 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.422705 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.422707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.422708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.422721 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.424250 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.424687 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.424689 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.424690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.424762 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.425003 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.425003 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.425004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.425127 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.425199 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.425519 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.425609 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:15:37.425836 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.414912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.18.166.193:43027)
0812 05:15:37.415134 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.415194 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.415966 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.416329 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.416334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.416688 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:37.416902 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.416907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.417873 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.417874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.418753 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.418756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.419262 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.419263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.419453 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.420233 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:37.420245 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:37.422552 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:37.424336 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.424343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.424345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.424588 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.424836 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.424837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.424837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.424943 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.425193 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:37.425198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:37.425325 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.425470 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.425627 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260812 05:15:37.426429 20485 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39129
I20260812 05:15:37.426564 20485 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:37.426784 20485 heartbeater.cc:511] Master 127.18.166.254:39129 requested a full tablet report, sending...
I20260812 05:15:37.427335 20267 ts_manager.cc:194] Registered new tserver with Master: 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701)
I20260812 05:15:37.428130 20267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:43027
I20260812 05:15:37.429559 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 20347
I20260812 05:15:37.429668 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:37.429886 19099 external_mini_cluster.cc:1428] Setting key a0c39c6984015e95e98b8489e91c8135
I20260812 05:15:37.434973 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.427680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.18.166.193:36093)
0812 05:15:37.427838 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.427849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.427974 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.428112 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.428113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.428311 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:37.428377 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.428379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.429382 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.429383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.430138 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.430141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.430640 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.430641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.430728 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.431274 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:37.431279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:37.431626 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:37.433827 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.433829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.433829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.434022 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.434241 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.434242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.434242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.434300 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.434611 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:37.434612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:37.434667 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.434724 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.434889 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260812 05:15:37.435037 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.427741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:36093 (local address 127.18.166.254:39129)
0812 05:15:37.427784 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.427786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.427956 (+ 170us) server_negotiation.cc:407] Connection header received
0812 05:15:37.428036 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.428037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.428048 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.428105 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.428469 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.429247 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.430232 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.430556 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.430642 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.431732 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.431733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.431734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.431743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.433720 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.434102 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.434103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.434104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.434171 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.434367 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.434368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.434368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.434538 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.434628 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.434774 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.434887 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:37.458786 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39129
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:37.545714 20494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:37.545989 20494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:37.546034 20494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:37.547302 20494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:37.547381 20494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:37.547417 20494 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:15:37.547461 20494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:37.549036 20494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:39129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:37.549346 20494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:37.549628 20494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:37.554991 20500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.554994 20503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.554991 20499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:37.555274 20494 server_base.cc:1034] running on GCE node
I20260812 05:15:37.555521 20494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:37.555743 20494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:37.556896 20494 hybrid_clock.cc:648] HybridClock initialized: now 1786511737556876 us; error 32 us; skew 500 ppm
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:37.559799 20494 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:37.560319 20494 webserver.cc:460] Webserver started at http://127.18.166.194:34703/ using document root <none> and password file <none>
I20260812 05:15:37.560592 20494 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:37.560667 20494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:37.560829 20494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:37.562028 20494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "ade76b77b46447bfab6734601e0a61ef"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "825fbaa03b2c1d6c4e59a39a78bb7923"
server_key_iv: "ee5b602b5f8eee834c4583be3ded0327"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.562438 20494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "ade76b77b46447bfab6734601e0a61ef"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "825fbaa03b2c1d6c4e59a39a78bb7923"
server_key_iv: "ee5b602b5f8eee834c4583be3ded0327"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.563926 20494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:37.564836 20510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.565201 20494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:37.565317 20494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "ade76b77b46447bfab6734601e0a61ef"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "825fbaa03b2c1d6c4e59a39a78bb7923"
server_key_iv: "ee5b602b5f8eee834c4583be3ded0327"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.565411 20494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:37.573108 20494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:37.575671 20494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:37.575860 20494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:37.576426 20494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:37.576491 20494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.576538 20494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:37.576568 20494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.584281 20494 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41727
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:37.584450 20630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41727 every 8 connection(s)
I20260812 05:15:37.584748 20494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:37.588061 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.577029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:55727 (local address 127.18.166.254:39129)
0812 05:15:37.577109 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.577112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.577331 (+ 219us) server_negotiation.cc:407] Connection header received
0812 05:15:37.577934 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.577936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.577951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.578004 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.578727 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.579547 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.581367 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.581759 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.581840 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.585061 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.585063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.585063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.585073 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.586617 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.587170 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.587170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.587171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.587230 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.587485 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.587486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.587486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.587602 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.587651 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.587921 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.587993 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:15:37.588306 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.576992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.18.166.194:55727)
0812 05:15:37.577183 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.577227 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.577804 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.578078 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.578083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.578390 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:37.578617 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.578622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.579675 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.579676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.581223 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.581228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.581902 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.581904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.582099 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.582755 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:37.582768 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:37.584934 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:37.586739 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.586748 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.586750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.587069 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.587294 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.587294 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.587295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.587411 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.587658 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:37.587663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:37.587803 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.587892 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.588031 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260812 05:15:37.588977 20631 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39129
I20260812 05:15:37.589123 20631 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:37.589351 20631 heartbeater.cc:511] Master 127.18.166.254:39129 requested a full tablet report, sending...
I20260812 05:15:37.589848 20267 ts_manager.cc:194] Registered new tserver with Master: ade76b77b46447bfab6734601e0a61ef (127.18.166.194:41727)
I20260812 05:15:37.590384 20267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:55727
I20260812 05:15:37.594636 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 20494
I20260812 05:15:37.594767 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:37.594961 19099 external_mini_cluster.cc:1428] Setting key a875908a11063746647389b052915309
I20260812 05:15:37.596591 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.590225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:36801 (local address 127.18.166.254:39129)
0812 05:15:37.590269 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.590271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.590406 (+ 135us) server_negotiation.cc:407] Connection header received
0812 05:15:37.590482 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.590484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.590494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.590548 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.590970 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.591539 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.592261 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.592443 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.592499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.593673 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.593674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.593674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.593680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.595138 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.595597 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.595598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.595598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.595681 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.595962 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.595963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.595963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.596133 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.596265 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.596441 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.596520 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:15:37.596763 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.590151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.18.166.194:36801)
0812 05:15:37.590336 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.590346 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.590422 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.590562 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.590563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.590813 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:37.590881 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.590884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.591638 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.591639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.592184 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.592185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.592518 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.592519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.592581 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.593145 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:37.593158 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:37.593589 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:37.595257 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.595258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.595258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.595488 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.595762 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.595764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.595764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.595855 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.596254 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:37.596255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:37.596334 (+ 79us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.596422 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.596584 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:37.623919 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39129
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:37.711479 20642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:37.711735 20642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:37.711766 20642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:37.713025 20642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:37.713078 20642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:37.713099 20642 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:15:37.713124 20642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:37.714931 20642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:39129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:37.715191 20642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:37.715406 20642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:37.720992 20650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:37.721161 20642 server_base.cc:1034] running on GCE node
W20260812 05:15:37.721040 20647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:37.721038 20648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:37.721572 20642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:37.721889 20642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:37.723042 20642 hybrid_clock.cc:648] HybridClock initialized: now 1786511737723016 us; error 34 us; skew 500 ppm
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:37.725859 20642 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:37.726370 20642 webserver.cc:460] Webserver started at http://127.18.166.195:35415/ using document root <none> and password file <none>
I20260812 05:15:37.726622 20642 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:37.726768 20642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:37.726980 20642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:37.728142 20642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "d8f92c43cc8e47eeb8f85d91f1726549"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "556885fe7b9e3287e542f28ce1b57502"
server_key_iv: "2eb1300ddfafc78498b9f0a4d1f3b8c0"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.728545 20642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "d8f92c43cc8e47eeb8f85d91f1726549"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "556885fe7b9e3287e542f28ce1b57502"
server_key_iv: "2eb1300ddfafc78498b9f0a4d1f3b8c0"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.730350 20642 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 05:15:37.731261 20658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.731477 20642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:37.731585 20642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "d8f92c43cc8e47eeb8f85d91f1726549"
format_stamp: "Formatted at 2026-08-12 05:15:37 on dist-test-slave-srgv"
server_key: "556885fe7b9e3287e542f28ce1b57502"
server_key_iv: "2eb1300ddfafc78498b9f0a4d1f3b8c0"
server_key_version: "encryptionkey@0"
I20260812 05:15:37.731675 20642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:37.764642 20642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:37.766855 20642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:37.767051 20642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:37.767617 20642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:37.767688 20642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:37.767743 20642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:37.767791 20642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:37.775743 20642 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:37121
I20260812 05:15:37.775841 20781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:37121 every 8 connection(s)
I20260812 05:15:37.776232 20642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:37.779023 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.768275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:58467 (local address 127.18.166.254:39129)
0812 05:15:37.768362 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.768365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.768632 (+ 267us) server_negotiation.cc:407] Connection header received
0812 05:15:37.769287 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.769289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.769306 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.769402 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.770250 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.770846 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.771934 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.772152 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.772237 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.775601 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.775603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.775604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.775613 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.777370 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.777934 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.777936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.777937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.778005 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.778292 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.778293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.778293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.778428 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.778504 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.778828 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.778926 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:15:37.779251 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.768249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.18.166.195:58467)
0812 05:15:37.768482 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.768527 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.769123 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.769543 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.769547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.769902 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:37.770115 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.770120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.770954 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.770955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.771828 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.771831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.772251 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.772252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.772449 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.773110 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:37.773121 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:37.775452 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:37.777547 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.777556 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.777558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.777831 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.778083 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.778084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.778084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.778213 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.778505 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:37.778514 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:37.778672 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.778785 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.778975 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":117,"threads_started":1}
I20260812 05:15:37.779847 20782 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39129
I20260812 05:15:37.779948 20782 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:37.780158 20782 heartbeater.cc:511] Master 127.18.166.254:39129 requested a full tablet report, sending...
I20260812 05:15:37.780689 20267 ts_manager.cc:194] Registered new tserver with Master: d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121)
I20260812 05:15:37.780990 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 20642
I20260812 05:15:37.781076 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:37.781224 20267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:58467
I20260812 05:15:37.781265 19099 external_mini_cluster.cc:1428] Setting key 7f42afd451b418adcf68d8a6cb9f5f28
I20260812 05:15:37.782198 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:15:37.788136 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.781156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:35859 (local address 127.18.166.254:39129)
0812 05:15:37.781228 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.781231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.781473 (+ 242us) server_negotiation.cc:407] Connection header received
0812 05:15:37.781564 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.781565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.781577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.781634 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.782068 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.782646 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.783766 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.783959 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.784017 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.785216 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.785218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.785218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.785227 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.786844 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.787254 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.787256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.787256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.787324 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.787542 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.787543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.787543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.787684 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.787765 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.787942 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.788033 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:15:37.788156 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.781244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.18.166.195:35859)
0812 05:15:37.781375 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.781390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.781504 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.781642 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.781644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.781890 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:37.781971 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.781974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.782762 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.782764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.783658 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.783662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.784041 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.784042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.784150 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.784763 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:37.784769 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:37.785098 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:37.786951 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.786953 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.786953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.787162 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.787404 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:37.787405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:37.787405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:37.787474 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:37.787755 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:37.787756 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:37.787815 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.787899 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.788035 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:15:37 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:37.808751 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.800815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.18.166.254:39129)
0812 05:15:37.800889 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.800891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.800902 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:15:37.800940 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.800941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.800951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.801023 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:37.801744 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.802252 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.803024 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.803228 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.803312 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.806215 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:37.806217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:37.806217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:37.806227 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:37.807580 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.807993 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.807993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.807994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.808048 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:37.808247 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:37.808248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:37.808248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:37.808365 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:37.808445 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.808563 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.808634 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:15:37.810542 20267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:15:37.812183 20267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:37.819710 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.814916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:37121 (local address 127.0.0.1:58184)
0812 05:15:37.815392 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.815443 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.815520 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.815856 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.815882 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.815884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:37.815974 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.815976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.817564 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.817565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.818330 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.818333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.819334 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.819335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.819430 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.819448 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.819527 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.819604 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":106,"threads_started":1}
I20260812 05:15:37.819737 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.815308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.18.166.195:37121)
0812 05:15:37.815486 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.815490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.815502 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:15:37.815660 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.815664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.815695 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.815769 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:37.816068 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.817208 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.818440 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.819226 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.819320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.819387 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.819561 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.819615 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260812 05:15:37.819941 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.815284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.18.166.193:43701)
0812 05:15:37.815571 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.815575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.815750 (+ 175us) server_negotiation.cc:407] Connection header received
0812 05:15:37.815797 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.815798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.815813 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.815892 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:37.816074 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.817215 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.818573 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.819301 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.819366 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.819415 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.819784 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.819834 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":124,"threads_started":1}
I20260812 05:15:37.819737 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.815130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:43701 (local address 127.0.0.1:58220)
0812 05:15:37.815646 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.815655 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.815702 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.815904 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.815905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.815907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:37.815975 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.815976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.817329 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.817330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.818461 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.818464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.819463 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.819464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.819550 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.819556 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.819609 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.819654 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":38,"thread_start_us":99,"threads_started":1}
I20260812 05:15:37.819814 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.815194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.18.166.194:41727)
0812 05:15:37.815376 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.815382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.815558 (+ 176us) server_negotiation.cc:407] Connection header received
0812 05:15:37.815633 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.815635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.815655 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.815779 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:37.816037 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.816960 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.818657 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.819265 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.819346 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.819402 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.819641 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.819684 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260812 05:15:37.820340 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.814916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41727 (local address 127.0.0.1:52572)
0812 05:15:37.815496 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.815506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.815560 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.815867 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.815874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.815877 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:37.815968 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.815971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.817054 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.817055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.818461 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.818464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.819468 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.819469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.819547 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.819550 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.820214 (+ 664us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.820271 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":15,"thread_start_us":61,"threads_started":1}
I20260812 05:15:37.820737 20713 tablet_service.cc:1467] Processing CreateTablet for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 (DEFAULT_TABLE table=test-table [id=eafb1e2c11cc4b1dbd15f5ecf688e71e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:37.820849 20416 tablet_service.cc:1467] Processing CreateTablet for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 (DEFAULT_TABLE table=test-table [id=eafb1e2c11cc4b1dbd15f5ecf688e71e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:37.821205 20713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:37.821305 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:37.821199 20564 tablet_service.cc:1467] Processing CreateTablet for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 (DEFAULT_TABLE table=test-table [id=eafb1e2c11cc4b1dbd15f5ecf688e71e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:37.821553 20564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:37.824604 20806 tablet_bootstrap.cc:492] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Bootstrap starting.
I20260812 05:15:37.824764 20807 tablet_bootstrap.cc:492] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Bootstrap starting.
I20260812 05:15:37.825428 20806 tablet_bootstrap.cc:654] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:37.825544 20808 tablet_bootstrap.cc:492] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Bootstrap starting.
I20260812 05:15:37.825663 20807 tablet_bootstrap.cc:654] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:37.826066 20807 log.cc:824] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:37.826081 20806 log.cc:824] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:37.826319 20808 tablet_bootstrap.cc:654] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:37.826697 20808 log.cc:824] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:37.826942 20806 tablet_bootstrap.cc:492] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: No bootstrap required, opened a new log
I20260812 05:15:37.827081 20806 ts_tablet_manager.cc:1397] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:37.827247 20807 tablet_bootstrap.cc:492] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: No bootstrap required, opened a new log
I20260812 05:15:37.827332 20807 ts_tablet_manager.cc:1397] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:37.828811 20807 raft_consensus.cc:348] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.828996 20807 raft_consensus.cc:374] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:37.829048 20807 raft_consensus.cc:729] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f92c43cc8e47eeb8f85d91f1726549, State: Initialized, Role: FOLLOWER
I20260812 05:15:37.828907 20806 raft_consensus.cc:348] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.829092 20806 raft_consensus.cc:374] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:37.829136 20806 raft_consensus.cc:729] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fac252c0ff43698257866ecd17decb, State: Initialized, Role: FOLLOWER
I20260812 05:15:37.829216 20807 consensus_queue.cc:260] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.829257 20806 consensus_queue.cc:260] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.829574 20782 heartbeater.cc:503] Master 127.18.166.254:39129 was elected leader, sending a full tablet report...
I20260812 05:15:37.829603 20806 ts_tablet_manager.cc:1428] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:15:37.829627 20485 heartbeater.cc:503] Master 127.18.166.254:39129 was elected leader, sending a full tablet report...
I20260812 05:15:37.829559 20807 ts_tablet_manager.cc:1428] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:15:37.830425 20808 tablet_bootstrap.cc:492] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: No bootstrap required, opened a new log
I20260812 05:15:37.830511 20808 ts_tablet_manager.cc:1397] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260812 05:15:37.832470 20808 raft_consensus.cc:348] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.832618 20808 raft_consensus.cc:374] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:37.832640 20808 raft_consensus.cc:729] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76b77b46447bfab6734601e0a61ef, State: Initialized, Role: FOLLOWER
I20260812 05:15:37.832739 20808 consensus_queue.cc:260] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.832978 20808 ts_tablet_manager.cc:1428] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:15:37.833055 20631 heartbeater.cc:503] Master 127.18.166.254:39129 was elected leader, sending a full tablet report...
W20260812 05:15:37.835510 20632 tablet.cc:2369] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:15:37.923921 20486 tablet.cc:2369] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:15:37.991789 20815 raft_consensus.cc:482] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:37.991927 20815 raft_consensus.cc:504] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.992287 20815 leader_election.cc:288] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121), 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701)
I20260812 05:15:37.996567 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.992594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:47531 (local address 127.18.166.195:37121)
0812 05:15:37.992659 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.992661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.992674 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:15:37.992711 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.992712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.992723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.992822 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:37.993031 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.993864 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.995499 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.996092 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.996180 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.996190 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.996440 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.996490 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:15:37.996567 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.992760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:41967 (local address 127.18.166.193:43701)
0812 05:15:37.992828 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:37.992830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:37.992929 (+ 99us) server_negotiation.cc:407] Connection header received
0812 05:15:37.993011 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:37.993012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:37.993022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:37.993086 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:37.993230 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.993936 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.995499 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.996083 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.996180 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.996190 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:37.996440 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 05:15:37.996487 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:15:37.996598 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.992509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:37121 (local address 127.18.166.194:47531)
0812 05:15:37.992577 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.992590 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.992643 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.992859 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.992860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.992862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:37.992953 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.992956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.993995 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.993996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.995370 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.995374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.996219 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.996220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.996355 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.996358 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.996414 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.996449 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":52}
I20260812 05:15:37.996608 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:37.992656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:43701 (local address 127.18.166.194:41967)
0812 05:15:37.992860 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:37.992871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:37.992929 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:37.993103 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:37.993104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:37.993105 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:37.993158 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.993160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.994016 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.994017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.995370 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:37.995374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:37.996219 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:37.996220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:37.996309 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:37.996314 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:15:37.996401 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:15:37.996448 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":32,"thread_start_us":122,"threads_started":1}
I20260812 05:15:37.997068 20734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a543da9ebcf4b5da4bd8240dd1f3fc1" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" is_pre_election: true
I20260812 05:15:37.997141 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a543da9ebcf4b5da4bd8240dd1f3fc1" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fac252c0ff43698257866ecd17decb" is_pre_election: true
I20260812 05:15:37.997293 20734 raft_consensus.cc:2454] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 0.
I20260812 05:15:37.997341 20438 raft_consensus.cc:2454] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 0.
I20260812 05:15:37.997579 20512 leader_election.cc:302] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ade76b77b46447bfab6734601e0a61ef, d8f92c43cc8e47eeb8f85d91f1726549; no voters:
I20260812 05:15:37.997743 20815 raft_consensus.cc:2781] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:37.997838 20815 raft_consensus.cc:482] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:37.997875 20815 raft_consensus.cc:3037] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:37.999136 20815 raft_consensus.cc:504] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:37.999315 20815 leader_election.cc:288] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 election: Requested vote from peers d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121), 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701)
I20260812 05:15:37.999557 20734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a543da9ebcf4b5da4bd8240dd1f3fc1" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f92c43cc8e47eeb8f85d91f1726549"
I20260812 05:15:37.999574 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a543da9ebcf4b5da4bd8240dd1f3fc1" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fac252c0ff43698257866ecd17decb"
I20260812 05:15:37.999675 20734 raft_consensus.cc:3037] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:37.999684 20438 raft_consensus.cc:3037] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.000730 20438 raft_consensus.cc:2454] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 1.
I20260812 05:15:38.000730 20734 raft_consensus.cc:2454] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 1.
I20260812 05:15:38.000955 20512 leader_election.cc:302] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ade76b77b46447bfab6734601e0a61ef, d8f92c43cc8e47eeb8f85d91f1726549; no voters:
I20260812 05:15:38.001068 20815 raft_consensus.cc:2781] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:38.001339 20815 raft_consensus.cc:686] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 1 LEADER]: Becoming Leader. State: Replica: ade76b77b46447bfab6734601e0a61ef, State: Running, Role: LEADER
I20260812 05:15:38.001510 20815 consensus_queue.cc:237] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } }
I20260812 05:15:38.002525 20266 catalog_manager.cc:5495] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef reported cstate change: term changed from 0 to 1, leader changed from <none> to ade76b77b46447bfab6734601e0a61ef (127.18.166.194). New cstate: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } health_report { overall_health: UNKNOWN } } }
W20260812 05:15:38.026925 20783 tablet.cc:2369] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:15:38.050380 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.047637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.18.166.194:41727)
0812 05:15:38.047713 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.047715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.047725 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:38.047806 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.047807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.047837 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.047911 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:38.048069 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.048560 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.049430 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.049628 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.049711 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.049965 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.050071 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.050167 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.050227 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.050252 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:15:38.052604 20438 raft_consensus.cc:1261] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Refusing update from remote peer ade76b77b46447bfab6734601e0a61ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:38.052634 20734 raft_consensus.cc:1261] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Refusing update from remote peer ade76b77b46447bfab6734601e0a61ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:38.053046 20815 consensus_queue.cc:1035] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:38.053252 20817 consensus_queue.cc:1035] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:38.056290 20820 mvcc.cc:205] Tried to move back new op lower bound from 7317552079059660800 to 7317552078854901760. Current Snapshot: MvccSnapshot[applied={T|T < 7317552079059660800}]
I20260812 05:15:38.056495 20822 mvcc.cc:205] Tried to move back new op lower bound from 7317552079059660800 to 7317552078854901760. Current Snapshot: MvccSnapshot[applied={T|T < 7317552079059660800}]
I20260812 05:15:38.057660 20821 mvcc.cc:205] Tried to move back new op lower bound from 7317552079059660800 to 7317552078854901760. Current Snapshot: MvccSnapshot[applied={T|T < 7317552079059660800 or (T in {7317552079059660800})}]
I20260812 05:15:38.061033 20266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:38.061172 20266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:38.062206 20266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: Sending DeleteTablet for 3 replicas of tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1
I20260812 05:15:38.062729 20564 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:52572
I20260812 05:15:38.062738 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58220
I20260812 05:15:38.063081 20830 tablet_replica.cc:330] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: stopping tablet replica
I20260812 05:15:38.063060 20266 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 05:15:38.063108 20713 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58184
I20260812 05:15:38.063211 20830 raft_consensus.cc:2229] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 05:15:38.063241 20266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:38.063341 20830 raft_consensus.cc:2258] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:38.064097 20830 ts_tablet_manager.cc:1905] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:38.064733 20416 tablet_service.cc:1467] Processing CreateTablet for tablet 61d82b8ab28c4351b153b5881f5b3f64 (DEFAULT_TABLE table=test-table [id=0691ec0e2c1f4d61a16889a8ce5294a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:38.064845 20564 tablet_service.cc:1467] Processing CreateTablet for tablet 61d82b8ab28c4351b153b5881f5b3f64 (DEFAULT_TABLE table=test-table [id=0691ec0e2c1f4d61a16889a8ce5294a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:38.064888 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d82b8ab28c4351b153b5881f5b3f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:38.064998 20564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d82b8ab28c4351b153b5881f5b3f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:38.065261 20713 tablet_service.cc:1467] Processing CreateTablet for tablet 61d82b8ab28c4351b153b5881f5b3f64 (DEFAULT_TABLE table=test-table [id=0691ec0e2c1f4d61a16889a8ce5294a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:38.065392 20713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d82b8ab28c4351b153b5881f5b3f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:38.066474 20806 tablet_bootstrap.cc:492] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Bootstrap starting.
I20260812 05:15:38.066541 20808 tablet_bootstrap.cc:492] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Bootstrap starting.
I20260812 05:15:38.066874 20807 tablet_bootstrap.cc:492] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Bootstrap starting.
I20260812 05:15:38.066963 20806 tablet_bootstrap.cc:654] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:38.066969 20808 tablet_bootstrap.cc:654] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:38.067318 20807 tablet_bootstrap.cc:654] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:38.067378 20830 ts_tablet_manager.cc:1918] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:15:38.067453 20830 log.cc:1196] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/1a543da9ebcf4b5da4bd8240dd1f3fc1
I20260812 05:15:38.067742 20830 ts_tablet_manager.cc:1939] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P 45fac252c0ff43698257866ecd17decb: Deleting consensus metadata
I20260812 05:15:38.068249 20252 catalog_manager.cc:4841] TS 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701): tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 (table test-table [id=eafb1e2c11cc4b1dbd15f5ecf688e71e]) successfully deleted
I20260812 05:15:38.068383 20806 tablet_bootstrap.cc:492] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: No bootstrap required, opened a new log
I20260812 05:15:38.068427 20806 ts_tablet_manager.cc:1397] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 05:15:38.068819 20806 raft_consensus.cc:348] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.068953 20806 raft_consensus.cc:374] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:38.068982 20806 raft_consensus.cc:729] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fac252c0ff43698257866ecd17decb, State: Initialized, Role: FOLLOWER
I20260812 05:15:38.069031 20806 consensus_queue.cc:260] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.069154 20806 ts_tablet_manager.cc:1428] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 05:15:38.069905 20808 tablet_bootstrap.cc:492] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: No bootstrap required, opened a new log
I20260812 05:15:38.069960 20807 tablet_bootstrap.cc:492] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: No bootstrap required, opened a new log
I20260812 05:15:38.069972 20808 ts_tablet_manager.cc:1397] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 05:15:38.070012 20807 ts_tablet_manager.cc:1397] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 05:15:38.070155 20808 raft_consensus.cc:348] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.070268 20832 tablet_replica.cc:330] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: stopping tablet replica
I20260812 05:15:38.070277 20808 raft_consensus.cc:374] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:38.070227 20807 raft_consensus.cc:348] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.070303 20808 raft_consensus.cc:729] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76b77b46447bfab6734601e0a61ef, State: Initialized, Role: FOLLOWER
I20260812 05:15:38.070307 20807 raft_consensus.cc:374] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:38.070328 20807 raft_consensus.cc:729] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f92c43cc8e47eeb8f85d91f1726549, State: Initialized, Role: FOLLOWER
I20260812 05:15:38.070354 20808 consensus_queue.cc:260] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.070370 20807 consensus_queue.cc:260] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.070394 20832 raft_consensus.cc:2229] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:38.070451 20808 ts_tablet_manager.cc:1428] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:38.070454 20807 ts_tablet_manager.cc:1428] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 05:15:38.077637 20831 tablet_replica.cc:330] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: stopping tablet replica
I20260812 05:15:38.077617 20832 raft_consensus.cc:2258] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:38.077939 20831 raft_consensus.cc:2229] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:38.078078 20831 raft_consensus.cc:2258] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:38.078299 20832 ts_tablet_manager.cc:1905] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:38.078601 20831 ts_tablet_manager.cc:1905] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:38.080574 20831 ts_tablet_manager.cc:1918] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:15:38.080574 20832 ts_tablet_manager.cc:1918] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:15:38.080679 20831 log.cc:1196] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/1a543da9ebcf4b5da4bd8240dd1f3fc1
I20260812 05:15:38.080681 20832 log.cc:1196] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/1a543da9ebcf4b5da4bd8240dd1f3fc1
I20260812 05:15:38.080964 20832 ts_tablet_manager.cc:1939] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P d8f92c43cc8e47eeb8f85d91f1726549: Deleting consensus metadata
I20260812 05:15:38.080979 20831 ts_tablet_manager.cc:1939] T 1a543da9ebcf4b5da4bd8240dd1f3fc1 P ade76b77b46447bfab6734601e0a61ef: Deleting consensus metadata
I20260812 05:15:38.081485 20253 catalog_manager.cc:4841] TS ade76b77b46447bfab6734601e0a61ef (127.18.166.194:41727): tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 (table test-table [id=eafb1e2c11cc4b1dbd15f5ecf688e71e]) successfully deleted
I20260812 05:15:38.081633 20252 catalog_manager.cc:4841] TS d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121): tablet 1a543da9ebcf4b5da4bd8240dd1f3fc1 (table test-table [id=eafb1e2c11cc4b1dbd15f5ecf688e71e]) successfully deleted
I20260812 05:15:38.397382 20812 raft_consensus.cc:482] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:38.397567 20812 raft_consensus.cc:504] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.397980 20812 leader_election.cc:288] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701), ade76b77b46447bfab6734601e0a61ef (127.18.166.194:41727)
I20260812 05:15:38.401340 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.398312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:53281 (local address 127.18.166.193:43701)
0812 05:15:38.398381 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.398384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.398446 (+ 62us) server_negotiation.cc:407] Connection header received
0812 05:15:38.398599 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.398600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.398611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.398674 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:38.398857 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.399320 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.400496 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.400951 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.401013 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.401020 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.401201 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.401258 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:15:38.401355 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.398213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:43701 (local address 127.18.166.195:53281)
0812 05:15:38.398371 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:38.398383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:38.398478 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:38.398691 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:38.398691 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:38.398693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:38.398774 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.398777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.399408 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.399410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.400403 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.400407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.401040 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.401042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.401121 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.401125 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:38.401184 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:15:38.401221 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":36,"thread_start_us":83,"threads_started":1}
I20260812 05:15:38.401510 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.398404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:55989 (local address 127.18.166.194:41727)
0812 05:15:38.398459 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.398462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.398471 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:38.398506 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.398508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.398522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.398582 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:38.398787 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.399384 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.400544 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.401028 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.401103 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.401113 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.401354 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.401432 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:15:38.401652 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d82b8ab28c4351b153b5881f5b3f64" candidate_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fac252c0ff43698257866ecd17decb" is_pre_election: true
I20260812 05:15:38.401757 20438 raft_consensus.cc:2454] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f92c43cc8e47eeb8f85d91f1726549 in term 0.
I20260812 05:15:38.401767 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.398274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41727 (local address 127.18.166.195:55989)
0812 05:15:38.398383 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:38.398396 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:38.398458 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:38.398616 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:38.398618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:38.398621 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:38.398709 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.398711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.399488 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.399488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.400424 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.400427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.401136 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.401137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.401235 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.401241 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:15:38.401300 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:15:38.401344 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":23,"thread_start_us":59,"threads_started":1}
I20260812 05:15:38.402051 20660 leader_election.cc:302] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45fac252c0ff43698257866ecd17decb, d8f92c43cc8e47eeb8f85d91f1726549; no voters:
I20260812 05:15:38.402236 20812 raft_consensus.cc:2781] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:38.402294 20812 raft_consensus.cc:482] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:38.402172 20585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d82b8ab28c4351b153b5881f5b3f64" candidate_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76b77b46447bfab6734601e0a61ef" is_pre_election: true
I20260812 05:15:38.402310 20812 raft_consensus.cc:3037] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.402354 20585 raft_consensus.cc:2454] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f92c43cc8e47eeb8f85d91f1726549 in term 0.
I20260812 05:15:38.403065 20812 raft_consensus.cc:504] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.403224 20812 leader_election.cc:288] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [CANDIDATE]: Term 1 election: Requested vote from peers 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701), ade76b77b46447bfab6734601e0a61ef (127.18.166.194:41727)
I20260812 05:15:38.403407 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d82b8ab28c4351b153b5881f5b3f64" candidate_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fac252c0ff43698257866ecd17decb"
I20260812 05:15:38.403419 20585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d82b8ab28c4351b153b5881f5b3f64" candidate_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76b77b46447bfab6734601e0a61ef"
I20260812 05:15:38.403497 20585 raft_consensus.cc:3037] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.403497 20438 raft_consensus.cc:3037] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.404382 20438 raft_consensus.cc:2454] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f92c43cc8e47eeb8f85d91f1726549 in term 1.
I20260812 05:15:38.404445 20585 raft_consensus.cc:2454] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f92c43cc8e47eeb8f85d91f1726549 in term 1.
I20260812 05:15:38.404577 20660 leader_election.cc:302] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45fac252c0ff43698257866ecd17decb, d8f92c43cc8e47eeb8f85d91f1726549; no voters:
I20260812 05:15:38.404731 20812 raft_consensus.cc:2781] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:38.404891 20812 raft_consensus.cc:686] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 LEADER]: Becoming Leader. State: Replica: d8f92c43cc8e47eeb8f85d91f1726549, State: Running, Role: LEADER
I20260812 05:15:38.404968 20812 consensus_queue.cc:237] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } }
I20260812 05:15:38.405711 20266 catalog_manager.cc:5495] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } health_report { overall_health: HEALTHY } } }
Aug 12 05:15:38 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:38.470117 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.462283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39129 (local address 127.0.0.1:42802)
0812 05:15:38.462522 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:38.462538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:38.462622 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:38.462826 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:38.462827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:38.463174 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:38.463236 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.463238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.463918 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.463918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.464483 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.464485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.464849 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.464850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.464895 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.465567 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:38.465573 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:38.467065 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:38.468934 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:38.468944 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:38.468946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:38.469182 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:38.469459 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:38.469460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:38.469460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:38.469530 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:38.469815 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:38.469823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:38.469872 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 05:15:38.469927 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:15:38.470038 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":116,"threads_started":1}
I20260812 05:15:38.470249 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.462374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42802 (local address 127.18.166.254:39129)
0812 05:15:38.462634 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.462636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.462645 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:38.462722 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.462723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.462735 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.462803 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:38.463320 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.463832 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.464569 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.464772 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.464829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.467169 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:38.467170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:38.467171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:38.467179 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:38.468845 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:38.469263 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:38.469264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:38.469264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:38.469365 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:38.469601 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:38.469602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:38.469602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:38.469750 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:38.469810 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.469961 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.470095 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":228,"thread_start_us":186,"threads_started":1}
I20260812 05:15:38.472643 20266 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:42802:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 05:15:38.473003 20266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:38.474677 20564 tablet_service.cc:1467] Processing CreateTablet for tablet dab1f21b25a54ad39ea5ec4a77d9a054 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=887af194ba284d58b27c2f36b47e849d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:15:38.474918 20564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab1f21b25a54ad39ea5ec4a77d9a054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:38.476192 20808 tablet_bootstrap.cc:492] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: Bootstrap starting.
I20260812 05:15:38.474684 20713 tablet_service.cc:1467] Processing CreateTablet for tablet dab1f21b25a54ad39ea5ec4a77d9a054 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=887af194ba284d58b27c2f36b47e849d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:15:38.476331 20416 tablet_service.cc:1467] Processing CreateTablet for tablet dab1f21b25a54ad39ea5ec4a77d9a054 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=887af194ba284d58b27c2f36b47e849d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:15:38.476419 20713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab1f21b25a54ad39ea5ec4a77d9a054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:38.476473 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab1f21b25a54ad39ea5ec4a77d9a054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:38.476588 20808 tablet_bootstrap.cc:654] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:38.477383 20808 tablet_bootstrap.cc:492] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: No bootstrap required, opened a new log
I20260812 05:15:38.477478 20808 ts_tablet_manager.cc:1397] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:38.477603 20808 raft_consensus.cc:348] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.477694 20808 raft_consensus.cc:374] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:38.477717 20808 raft_consensus.cc:729] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76b77b46447bfab6734601e0a61ef, State: Initialized, Role: FOLLOWER
I20260812 05:15:38.477787 20808 consensus_queue.cc:260] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.477907 20808 ts_tablet_manager.cc:1428] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:38.478026 20815 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.478099 20815 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.478109 20807 tablet_bootstrap.cc:492] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: Bootstrap starting.
I20260812 05:15:38.478139 20806 tablet_bootstrap.cc:492] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: Bootstrap starting.
I20260812 05:15:38.478583 20806 tablet_bootstrap.cc:654] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:38.478613 20807 tablet_bootstrap.cc:654] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:38.479429 20806 tablet_bootstrap.cc:492] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: No bootstrap required, opened a new log
I20260812 05:15:38.479496 20806 ts_tablet_manager.cc:1397] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 05:15:38.479632 20807 tablet_bootstrap.cc:492] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: No bootstrap required, opened a new log
I20260812 05:15:38.479677 20807 ts_tablet_manager.cc:1397] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:15:38.479655 20806 raft_consensus.cc:348] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.479753 20806 raft_consensus.cc:374] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:38.479785 20806 raft_consensus.cc:729] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fac252c0ff43698257866ecd17decb, State: Initialized, Role: FOLLOWER
I20260812 05:15:38.479840 20806 consensus_queue.cc:260] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.479856 20807 raft_consensus.cc:348] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.479934 20806 ts_tablet_manager.cc:1428] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:38.479944 20807 raft_consensus.cc:374] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:38.479969 20807 raft_consensus.cc:729] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f92c43cc8e47eeb8f85d91f1726549, State: Initialized, Role: FOLLOWER
I20260812 05:15:38.480020 20807 consensus_queue.cc:260] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.480037 20814 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.480131 20807 ts_tablet_manager.cc:1428] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:38.480139 20814 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.480208 20812 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.480319 20812 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.685762 20815 raft_consensus.cc:482] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:38.685886 20815 raft_consensus.cc:504] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.686021 20815 leader_election.cc:288] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701), d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121)
I20260812 05:15:38.686369 20734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab1f21b25a54ad39ea5ec4a77d9a054" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" is_pre_election: true
I20260812 05:15:38.686370 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab1f21b25a54ad39ea5ec4a77d9a054" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fac252c0ff43698257866ecd17decb" is_pre_election: true
I20260812 05:15:38.686489 20438 raft_consensus.cc:2454] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 0.
I20260812 05:15:38.686489 20734 raft_consensus.cc:2454] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 0.
I20260812 05:15:38.686666 20512 leader_election.cc:302] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ade76b77b46447bfab6734601e0a61ef, d8f92c43cc8e47eeb8f85d91f1726549; no voters:
I20260812 05:15:38.686784 20815 raft_consensus.cc:2781] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:38.686830 20815 raft_consensus.cc:482] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:38.686862 20815 raft_consensus.cc:3037] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.687583 20815 raft_consensus.cc:504] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.687734 20815 leader_election.cc:288] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 election: Requested vote from peers 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701), d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121)
I20260812 05:15:38.687956 20734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab1f21b25a54ad39ea5ec4a77d9a054" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f92c43cc8e47eeb8f85d91f1726549"
I20260812 05:15:38.687958 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab1f21b25a54ad39ea5ec4a77d9a054" candidate_uuid: "ade76b77b46447bfab6734601e0a61ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45fac252c0ff43698257866ecd17decb"
I20260812 05:15:38.688050 20734 raft_consensus.cc:3037] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.688050 20438 raft_consensus.cc:3037] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:38.688843 20734 raft_consensus.cc:2454] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 1.
I20260812 05:15:38.688869 20438 raft_consensus.cc:2454] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade76b77b46447bfab6734601e0a61ef in term 1.
I20260812 05:15:38.689069 20512 leader_election.cc:302] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45fac252c0ff43698257866ecd17decb, ade76b77b46447bfab6734601e0a61ef; no voters:
I20260812 05:15:38.689199 20815 raft_consensus.cc:2781] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:38.689333 20815 raft_consensus.cc:686] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [term 1 LEADER]: Becoming Leader. State: Replica: ade76b77b46447bfab6734601e0a61ef, State: Running, Role: LEADER
I20260812 05:15:38.689386 20815 consensus_queue.cc:237] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } }
I20260812 05:15:38.689729 20851 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: TxnStatusTablet state changed. Reason: New leader ade76b77b46447bfab6734601e0a61ef. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.689829 20851 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:15:38.690095 20853 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 05:15:38.690255 20853 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 05:15:38.690146 20267 catalog_manager.cc:5495] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef reported cstate change: term changed from 0 to 1, leader changed from <none> to ade76b77b46447bfab6734601e0a61ef (127.18.166.194). New cstate: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } health_report { overall_health: HEALTHY } } }
I20260812 05:15:38.708355 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.705564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41727 (local address 127.0.0.1:52600)
0812 05:15:38.705666 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:38.705677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:38.705731 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:38.705887 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:38.705888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:38.705889 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:38.705938 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.705940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.706861 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.706862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.707443 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.707445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.707929 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.707930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.707981 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.708001 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.708201 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.708210 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 05:15:38.708248 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 05:15:38.708291 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 05:15:38.708367 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.705624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.18.166.194:41727)
0812 05:15:38.705677 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.705679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.705715 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:15:38.705791 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.705792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.705804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.705880 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:38.705979 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.706744 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.707548 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.707842 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.707919 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.708095 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.708136 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.708191 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.708253 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.708279 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:15:38.709812 20438 raft_consensus.cc:1261] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Refusing update from remote peer ade76b77b46447bfab6734601e0a61ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:38.709813 20734 raft_consensus.cc:1261] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Refusing update from remote peer ade76b77b46447bfab6734601e0a61ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:38.710078 20815 consensus_queue.cc:1035] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:38.710232 20851 consensus_queue.cc:1035] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:38.710786 20814 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: TxnStatusTablet state changed. Reason: New leader ade76b77b46447bfab6734601e0a61ef. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.710887 20814 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.710834 20812 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: TxnStatusTablet state changed. Reason: New leader ade76b77b46447bfab6734601e0a61ef. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.710922 20812 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.712049 20812 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.712137 20812 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P d8f92c43cc8e47eeb8f85d91f1726549: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.712069 20815 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.712178 20815 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:15:38.712224 20851 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.712288 20851 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P ade76b77b46447bfab6734601e0a61ef: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:15:38.712991 20814 tablet_replica.cc:439] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ade76b77b46447bfab6734601e0a61ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 } } peers { permanent_uuid: "d8f92c43cc8e47eeb8f85d91f1726549" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 37121 } } peers { permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 } } }
I20260812 05:15:38.713104 20814 tablet_replica.cc:442] T dab1f21b25a54ad39ea5ec4a77d9a054 P 45fac252c0ff43698257866ecd17decb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:38.718945 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.716083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.18.166.195:37121)
0812 05:15:38.716221 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.716223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.716232 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:38.716288 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.716289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.716309 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.716384 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:38.716550 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.717040 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.718130 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.718348 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.718407 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.718590 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.718703 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.718762 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.718835 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.718856 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":97,"threads_started":1}
I20260812 05:15:38.724754 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.721796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:44061 (local address 127.18.166.194:41727)
0812 05:15:38.721861 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.721867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.721990 (+ 123us) server_negotiation.cc:407] Connection header received
0812 05:15:38.722056 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.722057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.722084 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.722138 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:38.722307 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.722772 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.723938 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.724136 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.724213 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.724519 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.724559 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.724602 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.724669 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.724690 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:15:38.724817 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.721714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41727 (local address 127.18.166.195:44061)
0812 05:15:38.721920 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:38.721932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:38.721994 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:38.722165 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:38.722166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:38.722167 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:38.722243 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.722245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.722855 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.722856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.723839 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.723842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.724243 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.724245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.724307 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.724326 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.724609 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.724611 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:15:38.724666 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:15:38.724715 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260812 05:15:38.727402 20438 raft_consensus.cc:1261] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Refusing update from remote peer d8f92c43cc8e47eeb8f85d91f1726549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:38.727442 20585 raft_consensus.cc:1261] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Refusing update from remote peer d8f92c43cc8e47eeb8f85d91f1726549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:38.727734 20838 consensus_queue.cc:1035] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fac252c0ff43698257866ecd17decb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:38.727905 20812 consensus_queue.cc:1035] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ade76b77b46447bfab6734601e0a61ef" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:38.738016 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.734772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:37121 (local address 127.18.166.194:36519)
0812 05:15:38.734964 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:38.734978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:38.735032 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:38.735208 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:38.735208 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:38.735210 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:38.735257 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.735259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.736157 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.736158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.737017 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:38.737021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.737508 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.737509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:38.737568 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.737598 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.737827 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.737830 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:38.737885 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:15:38.737938 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260812 05:15:38.737999 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.734979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:36519 (local address 127.18.166.195:37121)
0812 05:15:38.735023 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.735025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.735034 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:38.735133 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.735134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.735145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.735205 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:38.735298 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.736065 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.737189 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.737382 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.737506 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.737715 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:38.737760 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:38.737825 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.737889 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.737914 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:15:38.748875 20266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:38.749027 20266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:38.749802 20266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c8b44205799488a945024be570e2922: Sending DeleteTablet for 3 replicas of tablet 61d82b8ab28c4351b153b5881f5b3f64
I20260812 05:15:38.750118 20564 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d82b8ab28c4351b153b5881f5b3f64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:52572
I20260812 05:15:38.750252 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d82b8ab28c4351b153b5881f5b3f64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58220
I20260812 05:15:38.750531 20878 tablet_replica.cc:330] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: stopping tablet replica
I20260812 05:15:38.750641 20878 raft_consensus.cc:2229] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:38.750797 20878 raft_consensus.cc:2258] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:38.751271 20878 ts_tablet_manager.cc:1905] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:38.751293 20713 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d82b8ab28c4351b153b5881f5b3f64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58184
I20260812 05:15:38.751840 20880 tablet_replica.cc:330] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: stopping tablet replica
I20260812 05:15:38.751916 20880 raft_consensus.cc:2229] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:38.752030 20880 raft_consensus.cc:2258] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:38.752509 20880 ts_tablet_manager.cc:1905] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:38.753923 20878 ts_tablet_manager.cc:1918] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:38.754045 20878 log.cc:1196] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/61d82b8ab28c4351b153b5881f5b3f64
I20260812 05:15:38.754387 20878 ts_tablet_manager.cc:1939] T 61d82b8ab28c4351b153b5881f5b3f64 P 45fac252c0ff43698257866ecd17decb: Deleting consensus metadata
I20260812 05:15:38.754583 20880 ts_tablet_manager.cc:1918] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:38.754650 20880 log.cc:1196] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/61d82b8ab28c4351b153b5881f5b3f64
I20260812 05:15:38.754881 20880 ts_tablet_manager.cc:1939] T 61d82b8ab28c4351b153b5881f5b3f64 P d8f92c43cc8e47eeb8f85d91f1726549: Deleting consensus metadata
I20260812 05:15:38.754993 20252 catalog_manager.cc:4841] TS 45fac252c0ff43698257866ecd17decb (127.18.166.193:43701): tablet 61d82b8ab28c4351b153b5881f5b3f64 (table test-table [id=0691ec0e2c1f4d61a16889a8ce5294a7]) successfully deleted
I20260812 05:15:38.755232 20252 catalog_manager.cc:4841] TS d8f92c43cc8e47eeb8f85d91f1726549 (127.18.166.195:37121): tablet 61d82b8ab28c4351b153b5881f5b3f64 (table test-table [id=0691ec0e2c1f4d61a16889a8ce5294a7]) successfully deleted
I20260812 05:15:38.755414 20877 tablet_replica.cc:330] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: stopping tablet replica
I20260812 05:15:38.755509 20877 raft_consensus.cc:2229] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:38.755576 20877 raft_consensus.cc:2258] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:38.756060 20877 ts_tablet_manager.cc:1905] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:38.757863 20877 ts_tablet_manager.cc:1918] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:38.757934 20877 log.cc:1196] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/61d82b8ab28c4351b153b5881f5b3f64
I20260812 05:15:38.758195 20877 ts_tablet_manager.cc:1939] T 61d82b8ab28c4351b153b5881f5b3f64 P ade76b77b46447bfab6734601e0a61ef: Deleting consensus metadata
I20260812 05:15:38.758544 20253 catalog_manager.cc:4841] TS ade76b77b46447bfab6734601e0a61ef (127.18.166.194:41727): tablet 61d82b8ab28c4351b153b5881f5b3f64 (table test-table [id=0691ec0e2c1f4d61a16889a8ce5294a7]) successfully deleted
Aug 12 05:15:38 dist-test-slave-srgv krb5kdc[20221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:38.765805 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.751680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.18.166.193:43701)
0812 05:15:38.751726 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.751728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.755824 (+ 4096us) server_negotiation.cc:407] Connection header received
0812 05:15:38.755906 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.755907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.755920 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.755967 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:38.756802 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.757536 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.758852 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.759068 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.759151 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.762499 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:38.762515 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:38.762516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:38.762526 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:38.764389 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:38.764812 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:38.764817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:38.764817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:38.764920 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:38.765144 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:38.765145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:38.765145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:38.765316 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:38.765527 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.765573 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.765678 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:15:38.766155 20478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58222
I20260812 05:15:38.774139 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:38.767197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.18.166.254:39129)
0812 05:15:38.767252 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:38.767254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:38.767426 (+ 172us) server_negotiation.cc:407] Connection header received
0812 05:15:38.767504 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:38.767505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:38.767518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:38.767597 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:38.768275 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.768940 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.769735 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:38.769957 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:38.770034 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:38.771372 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:38.771374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:38.771374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:38.771383 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:38.772867 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:38.773252 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:38.773253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:38.773254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:38.773320 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:38.773554 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:38.773555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:38.773556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:38.773692 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:38.773806 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:38.773933 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 05:15:38.774028 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 05:15:38.774521 20266 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42804
I20260812 05:15:38.775279 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 20347
I20260812 05:15:38.789152 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 20494
I20260812 05:15:38.798173 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 20642
I20260812 05:15:38.805336 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 20235
2026-08-12T05:15:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2942 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:38 dist-test-slave-srgv krb5kdc[20886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:38 dist-test-slave-srgv krb5kdc[20886](info): set up 2 sockets
Aug 12 05:15:38 dist-test-slave-srgv krb5kdc[20886](info): commencing operation
krb5kdc: starting...
W20260812 05:15:39.879230 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:39 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:40.020089 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36609
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:36609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:40.120136 20900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:40.120523 20900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:40.120574 20900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:40.123256 20900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:40.123363 20900 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 05:15:40.123392 20900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:40.123420 20900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:40.123453 20900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:40.123488 20900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:40.126930 20900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:36609
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:40.127341 20900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:40.127665 20900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:40.133972 20906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:40.134148 20900 server_base.cc:1034] running on GCE node
W20260812 05:15:40.133972 20905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.134073 20908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:40.134591 20900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:40.134981 20900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:40.136200 20900 hybrid_clock.cc:648] HybridClock initialized: now 1786511740136169 us; error 53 us; skew 500 ppm
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:40.139297 20900 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:40.139868 20900 webserver.cc:460] Webserver started at http://127.18.166.254:37045/ using document root <none> and password file <none>
I20260812 05:15:40.140082 20900 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:40.140127 20900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:40.140241 20900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:40.141489 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "497826d69f014548b816ccd1735681d0"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "f89e6e3e3383000630274b90d5df6f04"
server_key_iv: "daace32480e8e88daaded45a1194ab1d"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.141862 20900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "497826d69f014548b816ccd1735681d0"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "f89e6e3e3383000630274b90d5df6f04"
server_key_iv: "daace32480e8e88daaded45a1194ab1d"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.143432 20900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:40.144304 20915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.144570 20900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:40.144671 20900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "497826d69f014548b816ccd1735681d0"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "f89e6e3e3383000630274b90d5df6f04"
server_key_iv: "daace32480e8e88daaded45a1194ab1d"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.144747 20900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:40.184773 20900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:40.187521 20900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:40.187655 20900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:40.192873 20900 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:36609
I20260812 05:15:40.193074 20980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:36609 every 8 connection(s)
I20260812 05:15:40.193367 20900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:40.194268 20981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.196555 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 20900
I20260812 05:15:40.196671 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:40.196841 19099 external_mini_cluster.cc:1428] Setting key d2b4441419a92a2c1a0d61bafff5452e
I20260812 05:15:40.197163 20981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Bootstrap starting.
I20260812 05:15:40.197988 20981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.198315 20981 log.cc:824] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:40.199108 20981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: No bootstrap required, opened a new log
I20260812 05:15:40.201140 20981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497826d69f014548b816ccd1735681d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36609 } }
I20260812 05:15:40.201355 20981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.201444 20981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 497826d69f014548b816ccd1735681d0, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.201637 20981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497826d69f014548b816ccd1735681d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36609 } }
I20260812 05:15:40.201749 20981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:40.201795 20981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:40.201841 20981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:40.202526 20981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497826d69f014548b816ccd1735681d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36609 } }
I20260812 05:15:40.202628 20981 leader_election.cc:302] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 497826d69f014548b816ccd1735681d0; no voters:
I20260812 05:15:40.202901 20981 leader_election.cc:288] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:40.202934 20986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:40.203059 20986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [term 1 LEADER]: Becoming Leader. State: Replica: 497826d69f014548b816ccd1735681d0, State: Running, Role: LEADER
I20260812 05:15:40.203184 20986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497826d69f014548b816ccd1735681d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36609 } }
I20260812 05:15:40.203222 20981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:40.203742 20988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 497826d69f014548b816ccd1735681d0. Latest consensus state: current_term: 1 leader_uuid: "497826d69f014548b816ccd1735681d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497826d69f014548b816ccd1735681d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36609 } } }
I20260812 05:15:40.203826 20988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:40.203989 20987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "497826d69f014548b816ccd1735681d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497826d69f014548b816ccd1735681d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36609 } } }
I20260812 05:15:40.204046 20987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:40.204197 20991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:40.205092 20991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:40.207132 20991 catalog_manager.cc:1347] Generated new cluster ID: f8ee6941f44e4ba1a1265f29e863012d
I20260812 05:15:40.207203 20991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:40.209506 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.197478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.18.166.254:36609)
0812 05:15:40.197709 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.197727 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.197757 (+ 30us) server_negotiation.cc:407] Connection header received
0812 05:15:40.198319 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.198327 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.198383 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.198575 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.199279 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.200213 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.201044 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.201644 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.201775 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.205084 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.205089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.205090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.205104 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.207638 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.208102 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.208106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.208107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.208217 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.208456 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.208457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.208457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.208737 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.208946 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.209029 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.209190 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":105,"threads_started":1}
I20260812 05:15:40.222939 20991 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:40.223601 20991 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:40.228830 20991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Generated new TSK 0
I20260812 05:15:40.229099 20991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:40.264704 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36609
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:40.356499 21007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:40.356767 21007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:40.356799 21007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:40.358198 21007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:40.358261 21007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:40.358285 21007 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 05:15:40.358297 21007 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:15:40.358315 21007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:40.360095 21007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:36609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:40.360399 21007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:40.360689 21007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:40.366084 21012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.366087 21015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.366114 21013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:40.366600 21007 server_base.cc:1034] running on GCE node
I20260812 05:15:40.366856 21007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:40.367129 21007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:40.368314 21007 hybrid_clock.cc:648] HybridClock initialized: now 1786511740368311 us; error 45 us; skew 500 ppm
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:40.371151 21007 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:40.371670 21007 webserver.cc:460] Webserver started at http://127.18.166.193:40369/ using document root <none> and password file <none>
I20260812 05:15:40.371914 21007 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:40.371985 21007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:40.372143 21007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:40.373308 21007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "0c16a748d4d5f0c13d90e3e70098f97f"
server_key_iv: "3e98c17458373bfbde79d8e41471cbf0"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.373785 21007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "0c16a748d4d5f0c13d90e3e70098f97f"
server_key_iv: "3e98c17458373bfbde79d8e41471cbf0"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.375386 21007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:40.376246 21022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.376457 21007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:40.376569 21007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "0c16a748d4d5f0c13d90e3e70098f97f"
server_key_iv: "3e98c17458373bfbde79d8e41471cbf0"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.376654 21007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:40.397773 21007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:40.400084 21007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:40.400285 21007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:40.400914 21007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:40.400976 21007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.401022 21007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:40.401062 21007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.408633 21007 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:41161
I20260812 05:15:40.408726 21142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:41161 every 8 connection(s)
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:40.409089 21007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:40.410975 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 21007
I20260812 05:15:40.411110 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:40.411331 19099 external_mini_cluster.cc:1428] Setting key 263c8d62feffdaeb17bac9cd2ab2d355
I20260812 05:15:40.412755 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.401477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:46921 (local address 127.18.166.254:36609)
0812 05:15:40.401549 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.401552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.401783 (+ 231us) server_negotiation.cc:407] Connection header received
0812 05:15:40.402321 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.402322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.402338 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.402430 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.403084 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.403928 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.405691 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.406096 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.406199 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.409669 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.409671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.409672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.409681 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.410942 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.411514 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.411515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.411516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.411591 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.411932 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.411934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.411934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.412110 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.412194 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.412582 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.412677 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:15:40.413059 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.401455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.18.166.193:46921)
0812 05:15:40.401631 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.401676 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.402203 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.402487 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.402491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.402763 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.402965 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.402970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.404084 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.404085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.405576 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.405580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.406336 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.406338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.406524 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.407238 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.407250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.409550 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.411055 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.411066 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.411069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.411402 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.411696 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.411697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.411698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.411841 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.412196 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.412205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.412408 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.412536 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.412747 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260812 05:15:40.414887 21143 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36609
I20260812 05:15:40.414995 21143 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:40.415279 21143 heartbeater.cc:511] Master 127.18.166.254:36609 requested a full tablet report, sending...
I20260812 05:15:40.415822 20933 ts_manager.cc:194] Registered new tserver with Master: a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161)
I20260812 05:15:40.416502 20933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:46921
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:15:40.423071 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.415055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:43195 (local address 127.18.166.254:36609)
0812 05:15:40.415101 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.415104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.415333 (+ 229us) server_negotiation.cc:407] Connection header received
0812 05:15:40.415372 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.415373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.415385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.415452 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.415988 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.416813 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.418155 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.418541 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.418633 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.419941 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.419942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.419943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.419953 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.421764 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.422307 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.422308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.422309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.422372 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.422615 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.422616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.422616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.422747 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.422803 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.422914 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.422998 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:15:40.423079 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.415044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.18.166.193:43195)
0812 05:15:40.415237 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.415250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.415323 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.415455 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.415456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.415800 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.415886 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.415889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.417020 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.417021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.418026 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.418030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.418644 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.418646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.418805 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.419304 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.419310 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.419819 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.421867 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.421869 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.421869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.422149 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.422457 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.422458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.422458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.422542 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.422793 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.422794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.422844 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.422894 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.423002 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:40.441231 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36609
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:40.530128 21153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:40.530355 21153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:40.530385 21153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:40.531534 21153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:40.531574 21153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:40.531594 21153 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 05:15:40.531605 21153 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:15:40.531622 21153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:40.533062 21153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:36609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:40.533281 21153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:40.533572 21153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:40.538890 21158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.538908 21161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.538920 21159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:40.539171 21153 server_base.cc:1034] running on GCE node
I20260812 05:15:40.539435 21153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:40.539760 21153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:40.540987 21153 hybrid_clock.cc:648] HybridClock initialized: now 1786511740540975 us; error 47 us; skew 500 ppm
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:40.543993 21153 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:40.544519 21153 webserver.cc:460] Webserver started at http://127.18.166.194:40703/ using document root <none> and password file <none>
I20260812 05:15:40.544785 21153 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:40.544839 21153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:40.545011 21153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:40.546311 21153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "3b8cda6c9b7a447ca785724e85b8648d"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "2412d881c812b47aef3e9a0f4cb0257e"
server_key_iv: "a39dd537856a78387e0e80c31481eede"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.546746 21153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "3b8cda6c9b7a447ca785724e85b8648d"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "2412d881c812b47aef3e9a0f4cb0257e"
server_key_iv: "a39dd537856a78387e0e80c31481eede"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.548508 21153 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 05:15:40.549394 21168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.549626 21153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:40.549768 21153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "3b8cda6c9b7a447ca785724e85b8648d"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "2412d881c812b47aef3e9a0f4cb0257e"
server_key_iv: "a39dd537856a78387e0e80c31481eede"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.549857 21153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:40.562376 21153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:40.564881 21153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:40.565083 21153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:40.565835 21153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:40.565902 21153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.565953 21153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:40.565980 21153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:40.574582 21153 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:33421
I20260812 05:15:40.574676 21287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:33421 every 8 connection(s)
I20260812 05:15:40.575048 21153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:40.577385 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 21153
I20260812 05:15:40.577548 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:40.577744 19099 external_mini_cluster.cc:1428] Setting key 0e38f2abe2389e50c514b025669a0f54
I20260812 05:15:40.578506 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.566317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:59091 (local address 127.18.166.254:36609)
0812 05:15:40.566397 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.566400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.566645 (+ 245us) server_negotiation.cc:407] Connection header received
0812 05:15:40.567250 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.567252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.567267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.567352 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.568621 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.569510 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.570638 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.570964 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.571039 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.574388 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.574390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.574391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.574401 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.576206 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.576662 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.576663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.576664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.576738 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.577011 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.577012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.577012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.577150 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.577234 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.578308 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.578409 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:15:40.578732 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.566311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.18.166.194:59091)
0812 05:15:40.566494 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.566540 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.567109 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.567447 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.567451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.567825 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.568392 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.568401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.569642 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.569643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.570511 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.570513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.571087 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.571088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.571281 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.572043 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.572055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.574240 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.576311 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.576319 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.576321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.576575 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.576817 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.576818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.576819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.576941 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.577937 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.577944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.578119 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.578283 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.578475 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260812 05:15:40.579344 21288 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36609
I20260812 05:15:40.579459 21288 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:40.579773 21288 heartbeater.cc:511] Master 127.18.166.254:36609 requested a full tablet report, sending...
I20260812 05:15:40.580173 20934 ts_manager.cc:194] Registered new tserver with Master: 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421)
I20260812 05:15:40.580668 20934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:59091
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:15:40.587373 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.580584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:50605 (local address 127.18.166.254:36609)
0812 05:15:40.580627 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.580629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.580790 (+ 161us) server_negotiation.cc:407] Connection header received
0812 05:15:40.580900 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.580901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.580914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.580946 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.581527 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.582017 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.583162 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.583354 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.583442 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.584522 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.584524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.584524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.584532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.586045 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.586440 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.586442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.586442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.586508 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.586725 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.586725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.586726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.586897 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.586978 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.587165 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.587250 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:15:40.587406 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.580524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.18.166.194:50605)
0812 05:15:40.580708 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.580724 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.580821 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.581016 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.581016 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.581296 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.581383 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.581386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.582141 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.582142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.583067 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.583071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.583444 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.583445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.583528 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.584087 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.584093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.584417 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.586164 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.586166 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.586166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.586360 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.586580 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.586581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.586581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.586647 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.586982 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.586984 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.587035 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.587120 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.587250 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:40.609205 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36609
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:40.697573 21297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:40.697813 21297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:40.697842 21297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:40.699193 21297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:40.699250 21297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:40.699275 21297 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 05:15:40.699327 21297 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:15:40.699354 21297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:40.701042 21297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:36609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:40.701331 21297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:40.701655 21297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:40.707472 21305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.707491 21302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:40.707486 21303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:40.707791 21297 server_base.cc:1034] running on GCE node
I20260812 05:15:40.708022 21297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:40.708276 21297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:40.709547 21297 hybrid_clock.cc:648] HybridClock initialized: now 1786511740709522 us; error 32 us; skew 500 ppm
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:40.712456 21297 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:40.713054 21297 webserver.cc:460] Webserver started at http://127.18.166.195:33657/ using document root <none> and password file <none>
I20260812 05:15:40.713330 21297 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:40.713383 21297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:40.713579 21297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:40.714815 21297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "144aa922b7ee4605b5f440efe4d66d70"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "cd7644ffbd729f4c8edda86489235677"
server_key_iv: "148b95174507e63b885959006f0686aa"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.715209 21297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "144aa922b7ee4605b5f440efe4d66d70"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "cd7644ffbd729f4c8edda86489235677"
server_key_iv: "148b95174507e63b885959006f0686aa"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.716773 21297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:15:40.717732 21314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.717932 21297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:40.718047 21297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "144aa922b7ee4605b5f440efe4d66d70"
format_stamp: "Formatted at 2026-08-12 05:15:40 on dist-test-slave-srgv"
server_key: "cd7644ffbd729f4c8edda86489235677"
server_key_iv: "148b95174507e63b885959006f0686aa"
server_key_version: "encryptionkey@0"
I20260812 05:15:40.718132 21297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:40.740690 21297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:40.743072 21297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:40.743270 21297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:40.743973 21297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:40.744081 21297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.744130 21297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:40.744175 21297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:40.752470 21297 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:33401
I20260812 05:15:40.752585 21436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:33401 every 8 connection(s)
I20260812 05:15:40.752985 21297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:40.754886 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.744475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:54239 (local address 127.18.166.254:36609)
0812 05:15:40.744553 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.744555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.744978 (+ 423us) server_negotiation.cc:407] Connection header received
0812 05:15:40.745923 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.745925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.745942 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.746013 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.746762 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.747286 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.748718 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.748931 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.749023 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.751641 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.751643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.751644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.751656 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.753752 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.754129 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.754130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.754130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.754189 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.754360 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.754360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.754360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.754470 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.754518 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.754746 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.754819 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:15:40.755143 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.744513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.18.166.195:54239)
0812 05:15:40.744814 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.744871 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.745711 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.746119 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.746124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.746427 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.746650 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.746655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.747397 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.747399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.748547 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.748550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.749035 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.749037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.749196 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.749858 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.749870 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.751504 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.753825 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.753829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.753831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.754066 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.754234 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.754234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.754234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.754323 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.754507 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.754510 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.754631 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.754718 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.754853 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":197,"threads_started":1}
I20260812 05:15:40.755414 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 21297
I20260812 05:15:40.755527 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:40.755657 21437 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36609
I20260812 05:15:40.755722 19099 external_mini_cluster.cc:1428] Setting key e75c6ed59758b566a4f7824ea3097c5d
I20260812 05:15:40.755774 21437 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:40.756024 21437 heartbeater.cc:511] Master 127.18.166.254:36609 requested a full tablet report, sending...
I20260812 05:15:40.756394 20934 ts_manager.cc:194] Registered new tserver with Master: 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:40.756804 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:15:40.756968 20934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:54239
I20260812 05:15:40.764027 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.756934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:59145 (local address 127.18.166.254:36609)
0812 05:15:40.756975 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.756978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.757109 (+ 131us) server_negotiation.cc:407] Connection header received
0812 05:15:40.757187 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.757188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.757197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.757237 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.757774 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.758422 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.759427 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.759650 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.759736 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.760944 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.760945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.760946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.760954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.762739 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.763228 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.763229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.763229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.763289 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.763513 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.763513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.763514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.763642 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.763697 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.763887 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.763955 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:15:40.764194 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.756865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.18.166.195:59145)
0812 05:15:40.757049 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.757061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.757123 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.757254 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.757255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.757540 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.757636 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.757638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.758543 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.758544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.759316 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.759319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.759735 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.759736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.759816 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.760324 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.760330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.760814 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.762864 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.762865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.762866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.763136 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.763365 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.763366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.763367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.763448 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.763708 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.763709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.763772 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.763867 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.764015 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:40.783898 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.776129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.18.166.254:36609)
0812 05:15:40.776195 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.776197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.776206 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:40.776234 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.776235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.776244 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.776312 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.776947 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.777559 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.778384 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.778595 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.778658 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.781386 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.781388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.781388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.781396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.782773 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.783146 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.783147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.783147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.783203 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.783437 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.783438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.783438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.783548 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.783626 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.783756 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.783825 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:15:40.785879 20934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:15:40.787626 20934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:40.795038 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.790683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33401 (local address 127.0.0.1:57204)
0812 05:15:40.791165 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.791174 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.791389 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.791573 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.791574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.791576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:40.791625 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.791627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.792461 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.792462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.793948 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.793950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.794692 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.794779 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.794794 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.794867 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.794933 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":58,"thread_start_us":81,"threads_started":1}
I20260812 05:15:40.795046 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.790413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.0.0.1:52492)
0812 05:15:40.790553 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.790606 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.790680 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.791137 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.791139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.791142 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:40.791225 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.791227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.792669 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.792670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.793908 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.793911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.794837 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.794897 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.794900 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.794957 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.794992 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":9,"thread_start_us":88,"threads_started":1}
I20260812 05:15:40.795140 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.790653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.18.166.193:41161)
0812 05:15:40.790863 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.790868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.790880 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:15:40.790928 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.790930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.790951 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.791031 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:40.791429 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.792578 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794029 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.794768 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794840 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.794890 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.794974 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.795025 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":100,"threads_started":1}
I20260812 05:15:40.795233 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.790542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.18.166.194:33421)
0812 05:15:40.790760 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.790767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.791024 (+ 257us) server_negotiation.cc:407] Connection header received
0812 05:15:40.791167 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.791170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.791186 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.791271 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:40.791420 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.792576 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.793905 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.794707 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794792 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.794854 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.795066 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.795121 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
I20260812 05:15:40.795434 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.790910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.18.166.195:33401)
0812 05:15:40.791368 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.791372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.791383 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:15:40.791461 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.791463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.791476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.791576 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:40.791683 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.792356 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794045 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.794584 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794695 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.794761 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.795243 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.795302 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":97,"threads_started":1}
I20260812 05:15:40.795481 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.790418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:33421 (local address 127.0.0.1:58238)
0812 05:15:40.790922 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.790933 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.791000 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.791273 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.791274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.791276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:40.791336 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.791338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.792688 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.792689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.793796 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.793800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.794895 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.794896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.794968 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.794973 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.795316 (+ 343us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.795368 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":107,"threads_started":1}
I20260812 05:15:40.796275 21075 tablet_service.cc:1467] Processing CreateTablet for tablet 9e72c0ba8cb4497ca14711e73e6529ca (DEFAULT_TABLE table=test-table [id=b5b9a807c5f54bdd9050986821e3ad7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:40.796272 21220 tablet_service.cc:1467] Processing CreateTablet for tablet 9e72c0ba8cb4497ca14711e73e6529ca (DEFAULT_TABLE table=test-table [id=b5b9a807c5f54bdd9050986821e3ad7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:40.796411 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 9e72c0ba8cb4497ca14711e73e6529ca (DEFAULT_TABLE table=test-table [id=b5b9a807c5f54bdd9050986821e3ad7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:40.796618 21075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e72c0ba8cb4497ca14711e73e6529ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.796664 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e72c0ba8cb4497ca14711e73e6529ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.796715 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e72c0ba8cb4497ca14711e73e6529ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.799947 21460 tablet_bootstrap.cc:492] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Bootstrap starting.
I20260812 05:15:40.800410 21462 tablet_bootstrap.cc:492] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Bootstrap starting.
I20260812 05:15:40.800647 21461 tablet_bootstrap.cc:492] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Bootstrap starting.
I20260812 05:15:40.800940 21460 tablet_bootstrap.cc:654] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.801045 21462 tablet_bootstrap.cc:654] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.801399 21460 log.cc:824] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:40.801438 21462 log.cc:824] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:40.801810 21461 tablet_bootstrap.cc:654] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.802186 21461 log.cc:824] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:40.802562 21462 tablet_bootstrap.cc:492] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: No bootstrap required, opened a new log
I20260812 05:15:40.802634 21460 tablet_bootstrap.cc:492] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: No bootstrap required, opened a new log
I20260812 05:15:40.802649 21462 ts_tablet_manager.cc:1397] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:15:40.802709 21460 ts_tablet_manager.cc:1397] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:40.803028 21461 tablet_bootstrap.cc:492] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: No bootstrap required, opened a new log
I20260812 05:15:40.803105 21461 ts_tablet_manager.cc:1397] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:15:40.804610 21460 raft_consensus.cc:348] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.804785 21460 raft_consensus.cc:374] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.804831 21460 raft_consensus.cc:729] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.804828 21462 raft_consensus.cc:348] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.805009 21462 raft_consensus.cc:374] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.805051 21462 raft_consensus.cc:729] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8cda6c9b7a447ca785724e85b8648d, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.804980 21460 consensus_queue.cc:260] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.805181 21462 consensus_queue.cc:260] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.805402 21460 ts_tablet_manager.cc:1428] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:15:40.805327 21461 raft_consensus.cc:348] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.805503 21461 raft_consensus.cc:374] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.805512 21462 ts_tablet_manager.cc:1428] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:15:40.805572 21143 heartbeater.cc:503] Master 127.18.166.254:36609 was elected leader, sending a full tablet report...
I20260812 05:15:40.805557 21461 raft_consensus.cc:729] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144aa922b7ee4605b5f440efe4d66d70, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.805647 21288 heartbeater.cc:503] Master 127.18.166.254:36609 was elected leader, sending a full tablet report...
I20260812 05:15:40.805713 21461 consensus_queue.cc:260] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.805980 21461 ts_tablet_manager.cc:1428] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 05:15:40.806031 21437 heartbeater.cc:503] Master 127.18.166.254:36609 was elected leader, sending a full tablet report...
I20260812 05:15:40.823562 21467 raft_consensus.cc:482] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:40.823716 21467 raft_consensus.cc:504] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.824064 21467 leader_election.cc:288] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401), a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161)
W20260812 05:15:40.826013 21289 tablet.cc:2369] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:15:40.827569 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.824393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:55595 (local address 127.18.166.193:41161)
0812 05:15:40.824453 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.824456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.824466 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:40.824498 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.824499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.824510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.824570 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:40.824795 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.825666 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.826837 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.827194 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.827288 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.827298 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.827451 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.827497 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:15:40.827600 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.824282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.18.166.194:55595)
0812 05:15:40.824331 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.824345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.824393 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.824608 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.824609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.824612 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:40.824703 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.824706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.825771 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.825772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.826726 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.826729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.827289 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.827290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.827370 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.827373 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.827443 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.827487 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":74}
I20260812 05:15:40.827630 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.824378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:49933 (local address 127.18.166.195:33401)
0812 05:15:40.824440 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.824442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.824780 (+ 338us) server_negotiation.cc:407] Connection header received
0812 05:15:40.824821 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.824823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.824832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.824891 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:40.825026 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.825834 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.826866 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.827257 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.827348 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.827356 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.827512 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.827552 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:15:40.827729 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.824284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33401 (local address 127.18.166.194:49933)
0812 05:15:40.824669 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.824681 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.824761 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.824901 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.824902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.824903 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:40.824958 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.824959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.825931 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.825932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.826762 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.826764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.827346 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.827348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.827432 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.827435 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.827504 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.827544 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":165,"thread_start_us":145,"threads_started":1}
I20260812 05:15:40.828080 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e72c0ba8cb4497ca14711e73e6529ca" candidate_uuid: "3b8cda6c9b7a447ca785724e85b8648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" is_pre_election: true
I20260812 05:15:40.828153 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e72c0ba8cb4497ca14711e73e6529ca" candidate_uuid: "3b8cda6c9b7a447ca785724e85b8648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70" is_pre_election: true
I20260812 05:15:40.828253 21096 raft_consensus.cc:2454] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b8cda6c9b7a447ca785724e85b8648d in term 0.
I20260812 05:15:40.828334 21388 raft_consensus.cc:2454] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b8cda6c9b7a447ca785724e85b8648d in term 0.
I20260812 05:15:40.828600 21170 leader_election.cc:302] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8cda6c9b7a447ca785724e85b8648d, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:40.828833 21467 raft_consensus.cc:2781] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:40.828879 21467 raft_consensus.cc:482] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:40.828895 21467 raft_consensus.cc:3037] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:40.829977 21467 raft_consensus.cc:504] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.830138 21467 leader_election.cc:288] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [CANDIDATE]: Term 1 election: Requested vote from peers 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401), a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161)
I20260812 05:15:40.830319 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e72c0ba8cb4497ca14711e73e6529ca" candidate_uuid: "3b8cda6c9b7a447ca785724e85b8648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70"
I20260812 05:15:40.830344 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e72c0ba8cb4497ca14711e73e6529ca" candidate_uuid: "3b8cda6c9b7a447ca785724e85b8648d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2"
I20260812 05:15:40.830433 21388 raft_consensus.cc:3037] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:40.830430 21096 raft_consensus.cc:3037] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:40.831148 21096 raft_consensus.cc:2454] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b8cda6c9b7a447ca785724e85b8648d in term 1.
I20260812 05:15:40.831274 21388 raft_consensus.cc:2454] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b8cda6c9b7a447ca785724e85b8648d in term 1.
I20260812 05:15:40.831332 21170 leader_election.cc:302] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8cda6c9b7a447ca785724e85b8648d, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:40.831414 21467 raft_consensus.cc:2781] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:40.831627 21467 raft_consensus.cc:686] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 1 LEADER]: Becoming Leader. State: Replica: 3b8cda6c9b7a447ca785724e85b8648d, State: Running, Role: LEADER
I20260812 05:15:40.831737 21467 consensus_queue.cc:237] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } }
I20260812 05:15:40.832625 20933 catalog_manager.cc:5495] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194). New cstate: current_term: 1 leader_uuid: "3b8cda6c9b7a447ca785724e85b8648d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } health_report { overall_health: UNKNOWN } } }
Aug 12 05:15:40 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:40.877707 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.870352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36609 (local address 127.0.0.1:57678)
0812 05:15:40.870614 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:40.870625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:40.870715 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:40.870871 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:40.870872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:40.871203 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:40.871252 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.871254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.871949 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.871950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.872472 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:40.872473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.872834 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.872835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:40.872887 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.873516 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:40.873520 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:40.874988 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:40.876704 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.876712 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.876714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.876915 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.877099 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:40.877099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:40.877099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:40.877155 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:40.877372 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:40.877378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:40.877474 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 05:15:40.877531 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:15:40.877632 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":141,"threads_started":1}
I20260812 05:15:40.877777 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:40.870457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.18.166.254:36609)
0812 05:15:40.870560 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:40.870563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:40.870693 (+ 130us) server_negotiation.cc:407] Connection header received
0812 05:15:40.870780 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:40.870780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:40.870796 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:40.870867 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:40.871366 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.871860 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.872572 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:40.872752 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:40.872835 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:40.875084 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:40.875085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:40.875085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:40.875092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:40.876624 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.876983 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.876984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.876984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.877037 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:40.877209 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:40.877209 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:40.877209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:40.877318 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:40.877368 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:40.877568 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 05:15:40.877682 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":71}
I20260812 05:15:40.879980 20933 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:57678:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 05:15:40.880249 20933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:40.881896 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 1a446a0d328d49138c55bfea8d14018a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9f52213608c4a778fd03ebc870b4432]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:15:40.881873 21075 tablet_service.cc:1467] Processing CreateTablet for tablet 1a446a0d328d49138c55bfea8d14018a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9f52213608c4a778fd03ebc870b4432]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:15:40.881896 21220 tablet_service.cc:1467] Processing CreateTablet for tablet 1a446a0d328d49138c55bfea8d14018a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a9f52213608c4a778fd03ebc870b4432]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:15:40.882086 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a446a0d328d49138c55bfea8d14018a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.882086 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a446a0d328d49138c55bfea8d14018a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.882087 21075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a446a0d328d49138c55bfea8d14018a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:40.883294 21461 tablet_bootstrap.cc:492] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: Bootstrap starting.
I20260812 05:15:40.883684 21462 tablet_bootstrap.cc:492] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: Bootstrap starting.
I20260812 05:15:40.883698 21460 tablet_bootstrap.cc:492] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: Bootstrap starting.
I20260812 05:15:40.883931 21461 tablet_bootstrap.cc:654] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.884081 21462 tablet_bootstrap.cc:654] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.884461 21460 tablet_bootstrap.cc:654] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:40.884807 21461 tablet_bootstrap.cc:492] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: No bootstrap required, opened a new log
I20260812 05:15:40.884828 21462 tablet_bootstrap.cc:492] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: No bootstrap required, opened a new log
I20260812 05:15:40.884857 21461 ts_tablet_manager.cc:1397] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 05:15:40.884877 21462 ts_tablet_manager.cc:1397] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:40.885013 21461 raft_consensus.cc:348] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:40.885097 21461 raft_consensus.cc:374] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.885026 21462 raft_consensus.cc:348] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:40.885118 21461 raft_consensus.cc:729] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144aa922b7ee4605b5f440efe4d66d70, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.885142 21462 raft_consensus.cc:374] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.885177 21462 raft_consensus.cc:729] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8cda6c9b7a447ca785724e85b8648d, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.885167 21461 consensus_queue.cc:260] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:40.885217 21462 consensus_queue.cc:260] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:40.885272 21460 tablet_bootstrap.cc:492] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: No bootstrap required, opened a new log
I20260812 05:15:40.885293 21461 ts_tablet_manager.cc:1428] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:40.885304 21462 ts_tablet_manager.cc:1428] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:15:40.885319 21460 ts_tablet_manager.cc:1397] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:15:40.885453 21467 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:40.885530 21467 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:40.885531 21468 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:40.885623 21468 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:40.885805 21460 raft_consensus.cc:348] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:40.885906 21460 raft_consensus.cc:374] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:40.885929 21460 raft_consensus.cc:729] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Initialized, Role: FOLLOWER
I20260812 05:15:40.885982 21460 consensus_queue.cc:260] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:40.886082 21460 ts_tablet_manager.cc:1428] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:40.886261 21466 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:40.886360 21466 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 05:15:40.909969 21144 tablet.cc:2369] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:15:41.004227 21438 tablet.cc:2369] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:15:41.113878 21466 raft_consensus.cc:482] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:41.113982 21466 raft_consensus.cc:504] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.114351 21466 leader_election.cc:288] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421), 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:41.117626 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.115001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:55791 (local address 127.18.166.195:33401)
0812 05:15:41.115048 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.115050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.115085 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:15:41.115204 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.115206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.115214 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.115288 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:41.115464 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.115927 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.116853 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.117214 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.117302 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.117312 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.117506 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.117549 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:15:41.117622 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.114776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:51647 (local address 127.18.166.194:33421)
0812 05:15:41.114849 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.114852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.114931 (+ 79us) server_negotiation.cc:407] Connection header received
0812 05:15:41.115052 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.115053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.115061 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.115088 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:41.115315 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.115793 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.116822 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.117207 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.117304 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.117313 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.117495 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.117537 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:15:41.117656 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.114666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:33421 (local address 127.18.166.193:51647)
0812 05:15:41.114838 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.114851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.114954 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.115171 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.115172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.115175 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:41.115244 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.115246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.115893 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.115894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.116710 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.116712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.117305 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.117306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.117390 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.117393 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.117488 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.117540 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260812 05:15:41.117664 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.114888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33401 (local address 127.18.166.193:55791)
0812 05:15:41.115029 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.115041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.115103 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.115307 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.115308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.115310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:41.115390 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.115391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.116019 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.116021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.116754 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.116756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.117303 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.117304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.117396 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.117400 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.117498 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.117553 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":3,"thread_start_us":50,"threads_started":1}
I20260812 05:15:41.118022 21241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a446a0d328d49138c55bfea8d14018a" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d" is_pre_election: true
I20260812 05:15:41.118180 21241 raft_consensus.cc:2454] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 0.
I20260812 05:15:41.118191 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a446a0d328d49138c55bfea8d14018a" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70" is_pre_election: true
I20260812 05:15:41.118299 21388 raft_consensus.cc:2454] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 0.
I20260812 05:15:41.118402 21026 leader_election.cc:302] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8cda6c9b7a447ca785724e85b8648d, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:41.118566 21466 raft_consensus.cc:2781] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:41.118631 21466 raft_consensus.cc:482] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:41.118665 21466 raft_consensus.cc:3037] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:41.119395 21466 raft_consensus.cc:504] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.119563 21466 leader_election.cc:288] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 election: Requested vote from peers 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421), 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:41.119784 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a446a0d328d49138c55bfea8d14018a" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70"
I20260812 05:15:41.119809 21241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a446a0d328d49138c55bfea8d14018a" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d"
I20260812 05:15:41.119882 21241 raft_consensus.cc:3037] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:41.119884 21388 raft_consensus.cc:3037] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:41.120577 21241 raft_consensus.cc:2454] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 1.
I20260812 05:15:41.120651 21388 raft_consensus.cc:2454] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 1.
I20260812 05:15:41.120795 21026 leader_election.cc:302] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8cda6c9b7a447ca785724e85b8648d, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:41.120888 21466 raft_consensus.cc:2781] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:41.121016 21466 raft_consensus.cc:686] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 LEADER]: Becoming Leader. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Running, Role: LEADER
I20260812 05:15:41.121130 21466 consensus_queue.cc:237] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.121731 21486 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: TxnStatusTablet state changed. Reason: New leader a3d9db3c9a5e4061a99f2249bfb6d0e2. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.121848 21486 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:15:41.122031 21488 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 05:15:41.122205 21488 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 05:15:41.122143 20934 catalog_manager.cc:5495] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193). New cstate: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } health_report { overall_health: UNKNOWN } } }
I20260812 05:15:41.174780 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.171650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.18.166.193:41161)
0812 05:15:41.171733 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.171735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.171847 (+ 112us) server_negotiation.cc:407] Connection header received
0812 05:15:41.171978 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.171978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.171998 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.172058 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.172231 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.172777 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.173919 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.174112 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.174174 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.174403 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.174517 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.174636 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.174691 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.174712 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:15:41.174834 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.171493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.0.0.1:52504)
0812 05:15:41.171774 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.171788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.171877 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.172090 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.172091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.172093 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:41.172160 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.172162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.172865 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.172866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.173823 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.173826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.174197 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.174198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.174272 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.174298 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.174617 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.174629 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.174681 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.174746 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260812 05:15:41.176923 21241 raft_consensus.cc:1261] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Refusing update from remote peer a3d9db3c9a5e4061a99f2249bfb6d0e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:41.176977 21388 raft_consensus.cc:1261] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Refusing update from remote peer a3d9db3c9a5e4061a99f2249bfb6d0e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:41.177322 21486 consensus_queue.cc:1035] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:41.177531 21466 consensus_queue.cc:1035] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:41.178740 21467 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: TxnStatusTablet state changed. Reason: New leader a3d9db3c9a5e4061a99f2249bfb6d0e2. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.178840 21467 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:41.178813 21468 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: TxnStatusTablet state changed. Reason: New leader a3d9db3c9a5e4061a99f2249bfb6d0e2. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.178934 21468 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:41.180117 21466 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.180226 21466 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:15:41.180732 21467 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.180836 21467 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P 3b8cda6c9b7a447ca785724e85b8648d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:41.181262 21486 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.181372 21486 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P a3d9db3c9a5e4061a99f2249bfb6d0e2: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:15:41.181970 21468 tablet_replica.cc:439] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } } }
I20260812 05:15:41.182109 21468 tablet_replica.cc:442] T 1a446a0d328d49138c55bfea8d14018a P 144aa922b7ee4605b5f440efe4d66d70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:15:41.182003 21493 mvcc.cc:205] Tried to move back new op lower bound from 7317552091857809408 to 7317552091632734208. Current Snapshot: MvccSnapshot[applied={T|T < 7317552091857809408}]
I20260812 05:15:41.186570 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.183929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.18.166.194:33421)
0812 05:15:41.183975 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.183978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.183987 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:41.184074 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.184075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.184093 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.184185 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.184339 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.184834 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.185725 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.185940 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.186018 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.186200 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.186319 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.186407 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.186458 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.186481 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:15:41.192239 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.189616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:52561 (local address 127.18.166.193:41161)
0812 05:15:41.189673 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.189675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.189796 (+ 121us) server_negotiation.cc:407] Connection header received
0812 05:15:41.189869 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.189870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.189878 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.189925 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.190074 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.190524 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.191559 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.191733 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.191785 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.192010 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.192039 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.192077 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.192162 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.192178 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:15:41.192312 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.189531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.18.166.194:52561)
0812 05:15:41.189718 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.189738 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.189807 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.189943 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.189944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.189946 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:41.190015 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.190016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.190608 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.190609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.191464 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.191467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.191809 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.191810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.191882 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.191902 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.192101 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.192103 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.192161 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.192208 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260812 05:15:41.195606 21388 raft_consensus.cc:1261] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Refusing update from remote peer 3b8cda6c9b7a447ca785724e85b8648d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:41.195919 21470 consensus_queue.cc:1035] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [LEADER]: Connected to new peer: Peer: permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:41.196452 21096 raft_consensus.cc:1261] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Refusing update from remote peer 3b8cda6c9b7a447ca785724e85b8648d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:41.196734 21467 consensus_queue.cc:1035] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:41.197944 21492 mvcc.cc:205] Tried to move back new op lower bound from 7317552091934441472 to 7317552090447515648. Current Snapshot: MvccSnapshot[applied={T|T < 7317552091934441472}]
I20260812 05:15:41.199352 21491 mvcc.cc:205] Tried to move back new op lower bound from 7317552091934441472 to 7317552090447515648. Current Snapshot: MvccSnapshot[applied={T|T < 7317552091934441472}]
I20260812 05:15:41.205958 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.202869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:52301 (local address 127.18.166.194:33421)
0812 05:15:41.202907 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.202909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.203072 (+ 163us) server_negotiation.cc:407] Connection header received
0812 05:15:41.203138 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.203139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.203150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.203199 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.203340 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.203989 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.205149 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.205367 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.205478 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.205671 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.205706 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.205787 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.205857 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.205879 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 05:15:41.206014 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.202815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:33421 (local address 127.18.166.193:52301)
0812 05:15:41.203000 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.203010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.203088 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.203200 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.203200 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.203202 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:41.203271 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.203272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.204134 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.204136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.205066 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.205070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.205504 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.205505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.205573 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.205588 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.205798 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.205800 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.205852 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.205897 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260812 05:15:41.214824 20932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:41.214944 20932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:41.215711 20932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Sending DeleteTablet for 3 replicas of tablet 9e72c0ba8cb4497ca14711e73e6529ca
I20260812 05:15:41.216049 21220 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e72c0ba8cb4497ca14711e73e6529ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:41 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260812 05:15:41.216081 21075 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e72c0ba8cb4497ca14711e73e6529ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:41 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:52492
I20260812 05:15:41.216252 20932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 05:15:41.216372 21368 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e72c0ba8cb4497ca14711e73e6529ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:41 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:57204
W20260812 05:15:41.216414 20932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:41.216625 21510 tablet_replica.cc:330] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: stopping tablet replica
I20260812 05:15:41.216751 21510 raft_consensus.cc:2229] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:41.216871 21510 raft_consensus.cc:2258] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:41.217038 21512 tablet_replica.cc:330] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: stopping tablet replica
I20260812 05:15:41.217161 21512 raft_consensus.cc:2229] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:41.217234 21512 raft_consensus.cc:2258] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:41.217875 21510 ts_tablet_manager.cc:1905] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:41.219102 21075 tablet_service.cc:1467] Processing CreateTablet for tablet 122268005c5f4ff187f4032e162b003e (DEFAULT_TABLE table=test-table [id=eadb9cd8c15b40c38026bf17095585bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:41.219172 21220 tablet_service.cc:1467] Processing CreateTablet for tablet 122268005c5f4ff187f4032e162b003e (DEFAULT_TABLE table=test-table [id=eadb9cd8c15b40c38026bf17095585bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:41.219260 21075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122268005c5f4ff187f4032e162b003e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:41.219310 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122268005c5f4ff187f4032e162b003e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:41.220274 21462 tablet_bootstrap.cc:492] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Bootstrap starting.
I20260812 05:15:41.220350 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 122268005c5f4ff187f4032e162b003e (DEFAULT_TABLE table=test-table [id=eadb9cd8c15b40c38026bf17095585bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:41.220489 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 122268005c5f4ff187f4032e162b003e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:41.220736 21462 tablet_bootstrap.cc:654] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:41.221530 21460 tablet_bootstrap.cc:492] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Bootstrap starting.
I20260812 05:15:41.222090 21461 tablet_bootstrap.cc:492] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Bootstrap starting.
I20260812 05:15:41.222162 21460 tablet_bootstrap.cc:654] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:41.222533 21461 tablet_bootstrap.cc:654] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:41.223186 21460 tablet_bootstrap.cc:492] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: No bootstrap required, opened a new log
I20260812 05:15:41.223249 21460 ts_tablet_manager.cc:1397] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:15:41.223457 21512 ts_tablet_manager.cc:1905] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:41.223443 21460 raft_consensus.cc:348] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.223521 21460 raft_consensus.cc:374] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:41.223547 21460 raft_consensus.cc:729] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Initialized, Role: FOLLOWER
I20260812 05:15:41.223589 21460 consensus_queue.cc:260] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.223707 21460 ts_tablet_manager.cc:1428] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:41.225960 21462 tablet_bootstrap.cc:492] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: No bootstrap required, opened a new log
I20260812 05:15:41.226023 21462 ts_tablet_manager.cc:1397] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 05:15:41.226150 21462 raft_consensus.cc:348] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.226210 21462 raft_consensus.cc:374] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:41.226224 21462 raft_consensus.cc:729] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8cda6c9b7a447ca785724e85b8648d, State: Initialized, Role: FOLLOWER
I20260812 05:15:41.226171 21511 tablet_replica.cc:330] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: stopping tablet replica
I20260812 05:15:41.226294 21462 consensus_queue.cc:260] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.226400 21462 ts_tablet_manager.cc:1428] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:15:41.226434 21511 raft_consensus.cc:2229] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:41.226521 21511 raft_consensus.cc:2258] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:41.225742 21512 ts_tablet_manager.cc:1918] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:41.227082 21512 log.cc:1196] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/9e72c0ba8cb4497ca14711e73e6529ca
I20260812 05:15:41.227370 21512 ts_tablet_manager.cc:1939] T 9e72c0ba8cb4497ca14711e73e6529ca P 144aa922b7ee4605b5f440efe4d66d70: Deleting consensus metadata
I20260812 05:15:41.227861 20920 catalog_manager.cc:4841] TS 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401): tablet 9e72c0ba8cb4497ca14711e73e6529ca (table test-table [id=b5b9a807c5f54bdd9050986821e3ad7f]) successfully deleted
I20260812 05:15:41.228569 21510 ts_tablet_manager.cc:1918] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:41.228673 21510 log.cc:1196] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/9e72c0ba8cb4497ca14711e73e6529ca
I20260812 05:15:41.228938 21510 ts_tablet_manager.cc:1939] T 9e72c0ba8cb4497ca14711e73e6529ca P 3b8cda6c9b7a447ca785724e85b8648d: Deleting consensus metadata
I20260812 05:15:41.229462 20920 catalog_manager.cc:4841] TS 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421): tablet 9e72c0ba8cb4497ca14711e73e6529ca (table test-table [id=b5b9a807c5f54bdd9050986821e3ad7f]) successfully deleted
I20260812 05:15:41.229764 21461 tablet_bootstrap.cc:492] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: No bootstrap required, opened a new log
I20260812 05:15:41.229848 21461 ts_tablet_manager.cc:1397] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20260812 05:15:41.230013 21461 raft_consensus.cc:348] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.230104 21461 raft_consensus.cc:374] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:41.230134 21461 raft_consensus.cc:729] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144aa922b7ee4605b5f440efe4d66d70, State: Initialized, Role: FOLLOWER
I20260812 05:15:41.230180 21461 consensus_queue.cc:260] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.230314 21461 ts_tablet_manager.cc:1428] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:41.233878 21511 ts_tablet_manager.cc:1905] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:41.235579 21511 ts_tablet_manager.cc:1918] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:41.235700 21511 log.cc:1196] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/9e72c0ba8cb4497ca14711e73e6529ca
I20260812 05:15:41.236313 21511 ts_tablet_manager.cc:1939] T 9e72c0ba8cb4497ca14711e73e6529ca P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting consensus metadata
I20260812 05:15:41.236819 20917 catalog_manager.cc:4841] TS a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161): tablet 9e72c0ba8cb4497ca14711e73e6529ca (table test-table [id=b5b9a807c5f54bdd9050986821e3ad7f]) successfully deleted
I20260812 05:15:41.285382 21468 raft_consensus.cc:482] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:41.285566 21468 raft_consensus.cc:504] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.285959 21468 leader_election.cc:288] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161), 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421)
I20260812 05:15:41.290459 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.286447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:58201 (local address 127.18.166.194:33421)
0812 05:15:41.286491 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.286493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.286619 (+ 126us) server_negotiation.cc:407] Connection header received
0812 05:15:41.286739 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.286740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.286765 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.286795 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:41.287070 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.287823 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.289470 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.290020 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.290073 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.290081 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.290347 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.290385 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:15:41.290459 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.286388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:56389 (local address 127.18.166.193:41161)
0812 05:15:41.286445 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.286447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.286620 (+ 173us) server_negotiation.cc:407] Connection header received
0812 05:15:41.286739 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.286740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.286751 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.286785 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:41.287069 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.287829 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.289468 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.290024 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.290073 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.290081 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.290334 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.290376 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:15:41.290805 21516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.286131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.18.166.195:56389)
0812 05:15:41.286544 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.286557 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.286652 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.286871 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.286872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.286875 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:41.286979 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.286981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.287939 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.287939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.289316 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.289320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.290153 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.290154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.290240 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.290246 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.290635 (+ 389us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.290706 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":63,"thread_start_us":60,"threads_started":1}
I20260812 05:15:41.291067 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122268005c5f4ff187f4032e162b003e" candidate_uuid: "144aa922b7ee4605b5f440efe4d66d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" is_pre_election: true
I20260812 05:15:41.291208 21096 raft_consensus.cc:2454] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144aa922b7ee4605b5f440efe4d66d70 in term 0.
I20260812 05:15:41.291186 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.286131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:33421 (local address 127.18.166.195:58201)
0812 05:15:41.286553 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.286564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.286652 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.286871 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.286872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.286874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:41.286978 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.286981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.287939 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.287940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.289317 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.289320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.290154 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.290155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.290240 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.290244 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.291058 (+ 814us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.291106 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":138,"thread_start_us":208,"threads_started":1}
I20260812 05:15:41.291450 21316 leader_election.cc:302] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144aa922b7ee4605b5f440efe4d66d70, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:41.291504 21241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122268005c5f4ff187f4032e162b003e" candidate_uuid: "144aa922b7ee4605b5f440efe4d66d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d" is_pre_election: true
I20260812 05:15:41.291610 21468 raft_consensus.cc:2781] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:41.291637 21241 raft_consensus.cc:2454] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144aa922b7ee4605b5f440efe4d66d70 in term 0.
I20260812 05:15:41.291646 21468 raft_consensus.cc:482] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:41.291671 21468 raft_consensus.cc:3037] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:41.292670 21468 raft_consensus.cc:504] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.292837 21468 leader_election.cc:288] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [CANDIDATE]: Term 1 election: Requested vote from peers a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161), 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421)
I20260812 05:15:41.293007 21241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122268005c5f4ff187f4032e162b003e" candidate_uuid: "144aa922b7ee4605b5f440efe4d66d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d"
I20260812 05:15:41.293044 21096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "122268005c5f4ff187f4032e162b003e" candidate_uuid: "144aa922b7ee4605b5f440efe4d66d70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2"
I20260812 05:15:41.293084 21241 raft_consensus.cc:3037] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:41.293114 21096 raft_consensus.cc:3037] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:41.293923 21096 raft_consensus.cc:2454] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144aa922b7ee4605b5f440efe4d66d70 in term 1.
I20260812 05:15:41.293941 21241 raft_consensus.cc:2454] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144aa922b7ee4605b5f440efe4d66d70 in term 1.
I20260812 05:15:41.294092 21316 leader_election.cc:302] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144aa922b7ee4605b5f440efe4d66d70, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:41.294173 21468 raft_consensus.cc:2781] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:41.294272 21468 raft_consensus.cc:686] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 1 LEADER]: Becoming Leader. State: Replica: 144aa922b7ee4605b5f440efe4d66d70, State: Running, Role: LEADER
I20260812 05:15:41.294335 21468 consensus_queue.cc:237] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:41.295017 20932 catalog_manager.cc:5495] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "144aa922b7ee4605b5f440efe4d66d70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } health_report { overall_health: HEALTHY } } }
I20260812 05:15:41.319936 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.317402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.18.166.195:33401)
0812 05:15:41.317492 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.317494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.317501 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:15:41.317541 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.317542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.317573 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.317652 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.317783 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.318274 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.319035 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.319255 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.319335 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.319529 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.319644 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.319731 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.319776 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.319802 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 05:15:41.326172 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.323153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.18.166.195:42219)
0812 05:15:41.323324 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.323336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.323396 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.323541 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.323542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.323544 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:41.323630 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.323632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.324572 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.324573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.325200 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.325202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.325735 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.325736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.325796 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.325815 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.326002 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.326003 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.326029 (+ 26us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.326091 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260812 05:15:41.326160 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.323288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:42219 (local address 127.18.166.193:41161)
0812 05:15:41.323378 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.323380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.323391 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:15:41.323448 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.323449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.323466 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.323539 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.323706 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.324470 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.325306 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.325645 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.325729 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.325908 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.325947 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.326005 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.326042 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.326071 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 05:15:41.328943 21096 raft_consensus.cc:1261] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Refusing update from remote peer 144aa922b7ee4605b5f440efe4d66d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:41.328981 21241 raft_consensus.cc:1261] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Refusing update from remote peer 144aa922b7ee4605b5f440efe4d66d70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:41.329291 21518 consensus_queue.cc:1035] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:41.329468 21468 consensus_queue.cc:1035] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:41.338721 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.336090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:52641 (local address 127.18.166.195:33401)
0812 05:15:41.336135 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:41.336137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:41.336143 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:15:41.336164 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:41.336165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:41.336176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:41.336228 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:41.336355 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.336919 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.337769 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.338048 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.338137 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.338388 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.338421 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.338516 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:41.338592 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 05:15:41.338620 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:15:41.338729 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:41.335884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33401 (local address 127.18.166.193:52641)
0812 05:15:41.336003 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:41.336018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:41.336075 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:41.336233 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:41.336234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:41.336235 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:41.336279 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.336281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.337014 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.337016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.337682 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:41.337685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:41.338197 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:41.338198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:41.338268 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:41.338279 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:41.338516 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:41.338518 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:15:41.338585 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:15:41.338646 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
W20260812 05:15:41.339154 21493 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 05:15:42.873445 20932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:42.873591 20932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:42.874617 20932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Sending DeleteTablet for 3 replicas of tablet 122268005c5f4ff187f4032e162b003e
I20260812 05:15:42.874953 21075 tablet_service.cc:1514] Processing DeleteTablet for tablet 122268005c5f4ff187f4032e162b003e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:42 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:52492
I20260812 05:15:42.874995 21368 tablet_service.cc:1514] Processing DeleteTablet for tablet 122268005c5f4ff187f4032e162b003e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:42 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:57204
I20260812 05:15:42.875020 21220 tablet_service.cc:1514] Processing DeleteTablet for tablet 122268005c5f4ff187f4032e162b003e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:42 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260812 05:15:42.875257 21551 tablet_replica.cc:330] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: stopping tablet replica
I20260812 05:15:42.875309 21551 raft_consensus.cc:2229] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:42.875340 21551 raft_consensus.cc:2258] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:42.875622 21551 ts_tablet_manager.cc:1905] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:42.877295 21551 ts_tablet_manager.cc:1918] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:15:42.877380 21551 log.cc:1196] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/122268005c5f4ff187f4032e162b003e
I20260812 05:15:42.877671 21551 ts_tablet_manager.cc:1939] T 122268005c5f4ff187f4032e162b003e P 3b8cda6c9b7a447ca785724e85b8648d: Deleting consensus metadata
I20260812 05:15:42.878234 21550 tablet_replica.cc:330] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: stopping tablet replica
I20260812 05:15:42.878321 21550 raft_consensus.cc:2229] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:42.878486 21550 raft_consensus.cc:2258] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:42.878603 20920 catalog_manager.cc:4841] TS 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421): tablet 122268005c5f4ff187f4032e162b003e (table test-table [id=eadb9cd8c15b40c38026bf17095585bb]) successfully deleted
I20260812 05:15:42.877858 21549 tablet_replica.cc:330] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: stopping tablet replica
I20260812 05:15:42.881304 21549 raft_consensus.cc:2229] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:42.881395 21549 raft_consensus.cc:2258] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:42.881726 21550 ts_tablet_manager.cc:1905] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:42.881906 21549 ts_tablet_manager.cc:1905] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:42.883610 21550 ts_tablet_manager.cc:1918] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:15:42.883718 21550 log.cc:1196] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/122268005c5f4ff187f4032e162b003e
I20260812 05:15:42.883721 21549 ts_tablet_manager.cc:1918] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:15:42.883837 21549 log.cc:1196] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/122268005c5f4ff187f4032e162b003e
I20260812 05:15:42.884032 21550 ts_tablet_manager.cc:1939] T 122268005c5f4ff187f4032e162b003e P 144aa922b7ee4605b5f440efe4d66d70: Deleting consensus metadata
I20260812 05:15:42.884179 21549 ts_tablet_manager.cc:1939] T 122268005c5f4ff187f4032e162b003e P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting consensus metadata
I20260812 05:15:42.884536 20920 catalog_manager.cc:4841] TS 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401): tablet 122268005c5f4ff187f4032e162b003e (table test-table [id=eadb9cd8c15b40c38026bf17095585bb]) successfully deleted
I20260812 05:15:42.884631 20917 catalog_manager.cc:4841] TS a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161): tablet 122268005c5f4ff187f4032e162b003e (table test-table [id=eadb9cd8c15b40c38026bf17095585bb]) successfully deleted
Aug 12 05:15:42 dist-test-slave-srgv krb5kdc[20886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:15:42 dist-test-slave-srgv krb5kdc[20886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:42.904714 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:42.895257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.18.166.254:36609)
0812 05:15:42.895445 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:42.895447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:42.895457 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:42.895486 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:42.895486 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:42.895497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:42.895579 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:42.896264 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:42.896900 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:42.898259 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:42.898490 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:42.898558 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:42.901531 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:42.901534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:42.901534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:42.901546 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:42.903303 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:42.903768 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:42.903770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:42.903770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:42.903875 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:42.904104 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:42.904105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:42.904105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:42.904327 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:42.904418 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:42.904547 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 05:15:42.904631 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260812 05:15:42.906518 20932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:15:42.906697 20932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:42.908305 21075 tablet_service.cc:1467] Processing CreateTablet for tablet 18b60d192f0a4794b3e64e346238c102 (DEFAULT_TABLE table=test-table [id=012407114b1f414a9170be11696cbb89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:42.908340 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 18b60d192f0a4794b3e64e346238c102 (DEFAULT_TABLE table=test-table [id=012407114b1f414a9170be11696cbb89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:42.908505 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b60d192f0a4794b3e64e346238c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:42.908505 21075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b60d192f0a4794b3e64e346238c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:42.910176 21561 tablet_bootstrap.cc:492] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Bootstrap starting.
I20260812 05:15:42.910216 21220 tablet_service.cc:1467] Processing CreateTablet for tablet 18b60d192f0a4794b3e64e346238c102 (DEFAULT_TABLE table=test-table [id=012407114b1f414a9170be11696cbb89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:42.910384 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b60d192f0a4794b3e64e346238c102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:42.910696 21561 tablet_bootstrap.cc:654] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:42.911058 21560 tablet_bootstrap.cc:492] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Bootstrap starting.
I20260812 05:15:42.911445 21561 tablet_bootstrap.cc:492] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: No bootstrap required, opened a new log
I20260812 05:15:42.911477 21560 tablet_bootstrap.cc:654] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:42.911643 21561 ts_tablet_manager.cc:1397] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:42.911918 21564 tablet_bootstrap.cc:492] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Bootstrap starting.
I20260812 05:15:42.911895 21561 raft_consensus.cc:348] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:42.912075 21561 raft_consensus.cc:374] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:42.912112 21561 raft_consensus.cc:729] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Initialized, Role: FOLLOWER
I20260812 05:15:42.912144 21560 tablet_bootstrap.cc:492] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: No bootstrap required, opened a new log
I20260812 05:15:42.912184 21561 consensus_queue.cc:260] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:42.912235 21560 ts_tablet_manager.cc:1397] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:42.912297 21564 tablet_bootstrap.cc:654] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:42.912364 21560 raft_consensus.cc:348] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:42.912461 21560 raft_consensus.cc:374] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:42.912465 21561 ts_tablet_manager.cc:1428] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 05:15:42.912485 21560 raft_consensus.cc:729] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144aa922b7ee4605b5f440efe4d66d70, State: Initialized, Role: FOLLOWER
I20260812 05:15:42.912545 21560 consensus_queue.cc:260] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:42.912663 21560 ts_tablet_manager.cc:1428] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:42.912978 21564 tablet_bootstrap.cc:492] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: No bootstrap required, opened a new log
I20260812 05:15:42.913038 21564 ts_tablet_manager.cc:1397] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:42.913151 21564 raft_consensus.cc:348] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:42.913239 21564 raft_consensus.cc:374] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:42.913273 21564 raft_consensus.cc:729] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8cda6c9b7a447ca785724e85b8648d, State: Initialized, Role: FOLLOWER
I20260812 05:15:42.913326 21564 consensus_queue.cc:260] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:42.913575 21564 ts_tablet_manager.cc:1428] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:43.071704 21498 raft_consensus.cc:482] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:43.071851 21498 raft_consensus.cc:504] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.072037 21498 leader_election.cc:288] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421), 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:43.072408 21240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b60d192f0a4794b3e64e346238c102" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d" is_pre_election: true
I20260812 05:15:43.072408 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b60d192f0a4794b3e64e346238c102" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70" is_pre_election: true
I20260812 05:15:43.072535 21388 raft_consensus.cc:2454] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 0.
I20260812 05:15:43.072535 21240 raft_consensus.cc:2454] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 0.
I20260812 05:15:43.072710 21026 leader_election.cc:302] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144aa922b7ee4605b5f440efe4d66d70, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:43.072835 21498 raft_consensus.cc:2781] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:43.072887 21498 raft_consensus.cc:482] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:43.072908 21498 raft_consensus.cc:3037] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:43.073824 21498 raft_consensus.cc:504] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.073993 21498 leader_election.cc:288] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 election: Requested vote from peers 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421), 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:43.074203 21240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b60d192f0a4794b3e64e346238c102" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d"
I20260812 05:15:43.074227 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b60d192f0a4794b3e64e346238c102" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70"
I20260812 05:15:43.074337 21388 raft_consensus.cc:3037] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:43.074342 21240 raft_consensus.cc:3037] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:43.075076 21388 raft_consensus.cc:2454] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 1.
I20260812 05:15:43.075157 21240 raft_consensus.cc:2454] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 1.
I20260812 05:15:43.075330 21026 leader_election.cc:302] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144aa922b7ee4605b5f440efe4d66d70, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:43.075482 21498 raft_consensus.cc:2781] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:43.075575 21498 raft_consensus.cc:686] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 LEADER]: Becoming Leader. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Running, Role: LEADER
I20260812 05:15:43.075644 21498 consensus_queue.cc:237] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.076385 20934 catalog_manager.cc:5495] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193). New cstate: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } health_report { overall_health: UNKNOWN } } }
I20260812 05:15:43.098378 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:43.095942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.18.166.193:41161)
0812 05:15:43.096107 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:43.096109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:43.096115 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:15:43.096136 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:43.096137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:43.096147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:43.096211 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:43.096380 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:43.096882 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:43.097715 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:43.097929 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:43.097991 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:43.098110 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:43.098142 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:43.098204 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:43.098245 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 05:15:43.098266 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260812 05:15:43.103718 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:43.101209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:50247 (local address 127.18.166.193:41161)
0812 05:15:43.101267 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:43.101269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:43.101279 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:43.101350 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:43.101351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:43.101363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:43.101461 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:43.101605 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:43.102105 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:43.102931 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:43.103117 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:43.103244 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:43.103415 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:43.103458 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:43.103521 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:43.103582 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 05:15:43.103609 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:15:43.103722 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:43.101028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.18.166.193:50247)
0812 05:15:43.101188 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:43.101200 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:43.101287 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:43.101478 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:43.101479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:43.101480 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:15:43.101537 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:43.101539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:43.102213 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:43.102215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:43.102819 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:43.102821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:43.103246 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:43.103247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:43.103306 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:43.103311 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:15:43.103521 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:15:43.103523 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:15:43.103578 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:15:43.103639 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260812 05:15:43.106369 21388 raft_consensus.cc:1261] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Refusing update from remote peer a3d9db3c9a5e4061a99f2249bfb6d0e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:43.106369 21240 raft_consensus.cc:1261] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Refusing update from remote peer a3d9db3c9a5e4061a99f2249bfb6d0e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:43.106753 21529 consensus_queue.cc:1035] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:43.106887 21498 consensus_queue.cc:1035] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:43.119342 20934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:43.119484 20934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:43.120260 20934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Sending DeleteTablet for 3 replicas of tablet 18b60d192f0a4794b3e64e346238c102
I20260812 05:15:43.120671 21220 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b60d192f0a4794b3e64e346238c102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:43 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260812 05:15:43.120671 21368 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b60d192f0a4794b3e64e346238c102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:43 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:57204
I20260812 05:15:43.120800 21551 tablet_replica.cc:330] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: stopping tablet replica
I20260812 05:15:43.120867 21551 raft_consensus.cc:2229] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:43.120925 21551 raft_consensus.cc:2258] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:43.121021 21550 tablet_replica.cc:330] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: stopping tablet replica
I20260812 05:15:43.121102 21550 raft_consensus.cc:2229] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:43.121160 21550 raft_consensus.cc:2258] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:43.121521 21550 ts_tablet_manager.cc:1905] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:43.121827 21551 ts_tablet_manager.cc:1905] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:43.123051 21075 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b60d192f0a4794b3e64e346238c102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:43 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:52492
I20260812 05:15:43.123255 21549 tablet_replica.cc:330] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: stopping tablet replica
I20260812 05:15:43.123327 21549 raft_consensus.cc:2229] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:43.123327 20934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 05:15:43.123489 21549 raft_consensus.cc:2258] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 05:15:43.123507 20934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:43.123519 21551 ts_tablet_manager.cc:1918] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:43.123584 21551 log.cc:1196] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/18b60d192f0a4794b3e64e346238c102
I20260812 05:15:43.123817 21551 ts_tablet_manager.cc:1939] T 18b60d192f0a4794b3e64e346238c102 P 3b8cda6c9b7a447ca785724e85b8648d: Deleting consensus metadata
I20260812 05:15:43.123968 21549 ts_tablet_manager.cc:1905] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:43.124104 21550 ts_tablet_manager.cc:1918] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:43.124163 21550 log.cc:1196] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/18b60d192f0a4794b3e64e346238c102
I20260812 05:15:43.124387 21550 ts_tablet_manager.cc:1939] T 18b60d192f0a4794b3e64e346238c102 P 144aa922b7ee4605b5f440efe4d66d70: Deleting consensus metadata
I20260812 05:15:43.124789 20920 catalog_manager.cc:4841] TS 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421): tablet 18b60d192f0a4794b3e64e346238c102 (table test-table [id=012407114b1f414a9170be11696cbb89]) successfully deleted
I20260812 05:15:43.124876 20920 catalog_manager.cc:4841] TS 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401): tablet 18b60d192f0a4794b3e64e346238c102 (table test-table [id=012407114b1f414a9170be11696cbb89]) successfully deleted
I20260812 05:15:43.125593 21549 ts_tablet_manager.cc:1918] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:15:43.125660 21549 log.cc:1196] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/18b60d192f0a4794b3e64e346238c102
I20260812 05:15:43.125898 21549 ts_tablet_manager.cc:1939] T 18b60d192f0a4794b3e64e346238c102 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Deleting consensus metadata
I20260812 05:15:43.126216 20917 catalog_manager.cc:4841] TS a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161): tablet 18b60d192f0a4794b3e64e346238c102 (table test-table [id=012407114b1f414a9170be11696cbb89]) successfully deleted
I20260812 05:15:43.126974 21220 tablet_service.cc:1467] Processing CreateTablet for tablet 8349910c92ab425db944a1228c72aa73 (DEFAULT_TABLE table=test-table [id=53e159be91ee468b9c4ff4501f18b2b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:43.127200 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8349910c92ab425db944a1228c72aa73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:43.128648 21368 tablet_service.cc:1467] Processing CreateTablet for tablet 8349910c92ab425db944a1228c72aa73 (DEFAULT_TABLE table=test-table [id=53e159be91ee468b9c4ff4501f18b2b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:43.128691 21564 tablet_bootstrap.cc:492] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Bootstrap starting.
I20260812 05:15:43.128820 21368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8349910c92ab425db944a1228c72aa73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:43.129101 21564 tablet_bootstrap.cc:654] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:43.129835 21564 tablet_bootstrap.cc:492] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: No bootstrap required, opened a new log
I20260812 05:15:43.129910 21564 ts_tablet_manager.cc:1397] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:43.130062 21564 raft_consensus.cc:348] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.130155 21564 raft_consensus.cc:374] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:43.130177 21564 raft_consensus.cc:729] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8cda6c9b7a447ca785724e85b8648d, State: Initialized, Role: FOLLOWER
I20260812 05:15:43.130234 21564 consensus_queue.cc:260] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.130338 21564 ts_tablet_manager.cc:1428] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 05:15:43.130980 21560 tablet_bootstrap.cc:492] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Bootstrap starting.
I20260812 05:15:43.131311 21075 tablet_service.cc:1467] Processing CreateTablet for tablet 8349910c92ab425db944a1228c72aa73 (DEFAULT_TABLE table=test-table [id=53e159be91ee468b9c4ff4501f18b2b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:43.131402 21560 tablet_bootstrap.cc:654] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:43.131446 21075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8349910c92ab425db944a1228c72aa73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:43.132061 21560 tablet_bootstrap.cc:492] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: No bootstrap required, opened a new log
I20260812 05:15:43.132112 21560 ts_tablet_manager.cc:1397] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:43.132264 21560 raft_consensus.cc:348] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.132349 21560 raft_consensus.cc:374] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:43.132375 21560 raft_consensus.cc:729] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144aa922b7ee4605b5f440efe4d66d70, State: Initialized, Role: FOLLOWER
I20260812 05:15:43.132433 21560 consensus_queue.cc:260] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.132531 21560 ts_tablet_manager.cc:1428] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:43.132972 21561 tablet_bootstrap.cc:492] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Bootstrap starting.
I20260812 05:15:43.133352 21561 tablet_bootstrap.cc:654] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:43.134042 21561 tablet_bootstrap.cc:492] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2: No bootstrap required, opened a new log
I20260812 05:15:43.134099 21561 ts_tablet_manager.cc:1397] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:43.134224 21561 raft_consensus.cc:348] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.134325 21561 raft_consensus.cc:374] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:43.134353 21561 raft_consensus.cc:729] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Initialized, Role: FOLLOWER
I20260812 05:15:43.134402 21561 consensus_queue.cc:260] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.134502 21561 ts_tablet_manager.cc:1428] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:43.216636 21575 raft_consensus.cc:482] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:43.216782 21575 raft_consensus.cc:504] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.216984 21575 leader_election.cc:288] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421), 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:43.217344 21240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8349910c92ab425db944a1228c72aa73" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d" is_pre_election: true
I20260812 05:15:43.217360 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8349910c92ab425db944a1228c72aa73" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70" is_pre_election: true
I20260812 05:15:43.217489 21240 raft_consensus.cc:2454] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 0.
I20260812 05:15:43.217489 21388 raft_consensus.cc:2454] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 0.
I20260812 05:15:43.217700 21026 leader_election.cc:302] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144aa922b7ee4605b5f440efe4d66d70, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:43.217810 21575 raft_consensus.cc:2781] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:43.217855 21575 raft_consensus.cc:482] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:43.217886 21575 raft_consensus.cc:3037] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:43.218690 21575 raft_consensus.cc:504] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.218858 21575 leader_election.cc:288] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 election: Requested vote from peers 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421), 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401)
I20260812 05:15:43.219067 21240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8349910c92ab425db944a1228c72aa73" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8cda6c9b7a447ca785724e85b8648d"
I20260812 05:15:43.219064 21388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8349910c92ab425db944a1228c72aa73" candidate_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144aa922b7ee4605b5f440efe4d66d70"
I20260812 05:15:43.219182 21240 raft_consensus.cc:3037] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:43.219180 21388 raft_consensus.cc:3037] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:43.219931 21240 raft_consensus.cc:2454] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 1.
I20260812 05:15:43.219969 21388 raft_consensus.cc:2454] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d9db3c9a5e4061a99f2249bfb6d0e2 in term 1.
I20260812 05:15:43.220165 21026 leader_election.cc:302] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144aa922b7ee4605b5f440efe4d66d70, a3d9db3c9a5e4061a99f2249bfb6d0e2; no voters:
I20260812 05:15:43.220340 21575 raft_consensus.cc:2781] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:43.220427 21575 raft_consensus.cc:686] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [term 1 LEADER]: Becoming Leader. State: Replica: a3d9db3c9a5e4061a99f2249bfb6d0e2, State: Running, Role: LEADER
I20260812 05:15:43.220475 21575 consensus_queue.cc:237] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } }
I20260812 05:15:43.221257 20932 catalog_manager.cc:5495] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193). New cstate: current_term: 1 leader_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3d9db3c9a5e4061a99f2249bfb6d0e2" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 41161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 } health_report { overall_health: UNKNOWN } } }
I20260812 05:15:43.227675 21241 raft_consensus.cc:1261] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Refusing update from remote peer a3d9db3c9a5e4061a99f2249bfb6d0e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:43.227681 21388 raft_consensus.cc:1261] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Refusing update from remote peer a3d9db3c9a5e4061a99f2249bfb6d0e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:43.228018 21529 consensus_queue.cc:1035] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144aa922b7ee4605b5f440efe4d66d70" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:43.228150 21498 consensus_queue.cc:1035] T 8349910c92ab425db944a1228c72aa73 P a3d9db3c9a5e4061a99f2249bfb6d0e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8cda6c9b7a447ca785724e85b8648d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 05:15:43.232937 21527 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 05:15:44.756105 20932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:44.756263 20932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:44.757306 20932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 497826d69f014548b816ccd1735681d0: Sending DeleteTablet for 3 replicas of tablet 8349910c92ab425db944a1228c72aa73
I20260812 05:15:44.757639 21075 tablet_service.cc:1514] Processing DeleteTablet for tablet 8349910c92ab425db944a1228c72aa73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:44 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:52492
I20260812 05:15:44.757660 21368 tablet_service.cc:1514] Processing DeleteTablet for tablet 8349910c92ab425db944a1228c72aa73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:44 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:57204
I20260812 05:15:44.757859 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 21007
I20260812 05:15:44.758389 21220 tablet_service.cc:1514] Processing DeleteTablet for tablet 8349910c92ab425db944a1228c72aa73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:44 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260812 05:15:44.759003 21606 tablet_replica.cc:330] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: stopping tablet replica
I20260812 05:15:44.759065 21606 raft_consensus.cc:2229] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:44.759110 21606 raft_consensus.cc:2258] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:44.759642 21606 ts_tablet_manager.cc:1905] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:44.762164 21606 ts_tablet_manager.cc:1918] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:15:44.762265 21606 log.cc:1196] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/8349910c92ab425db944a1228c72aa73
I20260812 05:15:44.762513 21606 ts_tablet_manager.cc:1939] T 8349910c92ab425db944a1228c72aa73 P 144aa922b7ee4605b5f440efe4d66d70: Deleting consensus metadata
I20260812 05:15:44.762893 21608 tablet_replica.cc:330] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: stopping tablet replica
I20260812 05:15:44.762971 21608 raft_consensus.cc:2229] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:44.762936 20920 catalog_manager.cc:4841] TS 144aa922b7ee4605b5f440efe4d66d70 (127.18.166.195:33401): tablet 8349910c92ab425db944a1228c72aa73 (table test-table [id=53e159be91ee468b9c4ff4501f18b2b9]) successfully deleted
I20260812 05:15:44.763022 21608 raft_consensus.cc:2258] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:44.763319 21608 ts_tablet_manager.cc:1905] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:44.765348 21608 ts_tablet_manager.cc:1918] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:15:44.765471 21608 log.cc:1196] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/8349910c92ab425db944a1228c72aa73
I20260812 05:15:44.765741 21608 ts_tablet_manager.cc:1939] T 8349910c92ab425db944a1228c72aa73 P 3b8cda6c9b7a447ca785724e85b8648d: Deleting consensus metadata
I20260812 05:15:44.766331 20920 catalog_manager.cc:4841] TS 3b8cda6c9b7a447ca785724e85b8648d (127.18.166.194:33421): tablet 8349910c92ab425db944a1228c72aa73 (table test-table [id=53e159be91ee468b9c4ff4501f18b2b9]) successfully deleted
W20260812 05:15:44.771039 20917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:41161 (error 108)
I20260812 05:15:44.771279 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 21153
I20260812 05:15:44.772756 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:44.772495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:41161 (local address 127.0.0.1:52528)
0812 05:15:44.772650 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:44.772710 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:41161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
W20260812 05:15:44.772892 20917 catalog_manager.cc:4584] TS a3d9db3c9a5e4061a99f2249bfb6d0e2 (127.18.166.193:41161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8349910c92ab425db944a1228c72aa73: Network error: Client connection negotiation failed: client connection to 127.18.166.193:41161: connect: Connection refused (error 111)
I20260812 05:15:44.779031 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 21297
I20260812 05:15:44.787050 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 20900
2026-08-12T05:15:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5982 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:44 dist-test-slave-srgv krb5kdc[21613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:44 dist-test-slave-srgv krb5kdc[21613](info): set up 2 sockets
Aug 12 05:15:44 dist-test-slave-srgv krb5kdc[21613](info): commencing operation
krb5kdc: starting...
W20260812 05:15:45.871515 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:45 dist-test-slave-srgv krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:46.000484 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:33461
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:46.088030 21626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:46.088347 21626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:46.088392 21626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:46.089771 21626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:46.089869 21626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:46.089898 21626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:46.089922 21626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:46.089949 21626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:46.091717 21626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:33461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:46.092060 21626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:46.092329 21626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:46.097507 21632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:46.097500 21636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:46.097500 21633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:46.097752 21626 server_base.cc:1034] running on GCE node
I20260812 05:15:46.098074 21626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:46.098444 21626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:46.099635 21626 hybrid_clock.cc:648] HybridClock initialized: now 1786511746099622 us; error 45 us; skew 500 ppm
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:46.107059 21626 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:46.107558 21626 webserver.cc:460] Webserver started at http://127.18.166.254:33529/ using document root <none> and password file <none>
I20260812 05:15:46.107785 21626 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:46.107832 21626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:46.107935 21626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:46.109072 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "de8ae17fbc2d46b189063274e602d4dc"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "14671da40823882f088bb09fdf0e6bdc"
server_key_iv: "6eee576c392907cdd883c16975f327aa"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.109483 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "de8ae17fbc2d46b189063274e602d4dc"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "14671da40823882f088bb09fdf0e6bdc"
server_key_iv: "6eee576c392907cdd883c16975f327aa"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.111016 21626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:46.111833 21643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.112039 21626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:46.112141 21626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "de8ae17fbc2d46b189063274e602d4dc"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "14671da40823882f088bb09fdf0e6bdc"
server_key_iv: "6eee576c392907cdd883c16975f327aa"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.112223 21626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:46.142896 21626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:46.145562 21626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:46.145701 21626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:46.150286 21626 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:33461
I20260812 05:15:46.150362 21697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:33461 every 8 connection(s)
I20260812 05:15:46.150777 21626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:46.151814 21698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:46.154556 21698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc: Bootstrap starting.
I20260812 05:15:46.155267 21698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:46.155613 21698 log.cc:824] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:46.156625 21698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc: No bootstrap required, opened a new log
I20260812 05:15:46.156842 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 21626
I20260812 05:15:46.156944 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:46.157181 19099 external_mini_cluster.cc:1428] Setting key 3e4d378e2209a20522a19ab5f52441f6
I20260812 05:15:46.159988 21698 raft_consensus.cc:348] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8ae17fbc2d46b189063274e602d4dc" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33461 } }
I20260812 05:15:46.160231 21698 raft_consensus.cc:374] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:46.160274 21698 raft_consensus.cc:729] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de8ae17fbc2d46b189063274e602d4dc, State: Initialized, Role: FOLLOWER
I20260812 05:15:46.160503 21698 consensus_queue.cc:260] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8ae17fbc2d46b189063274e602d4dc" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33461 } }
I20260812 05:15:46.160632 21698 raft_consensus.cc:388] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:46.160681 21698 raft_consensus.cc:482] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:46.160730 21698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:46.161681 21698 raft_consensus.cc:504] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8ae17fbc2d46b189063274e602d4dc" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33461 } }
I20260812 05:15:46.161803 21698 leader_election.cc:302] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de8ae17fbc2d46b189063274e602d4dc; no voters:
I20260812 05:15:46.162009 21698 leader_election.cc:288] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:46.162070 21703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:46.162253 21703 raft_consensus.cc:686] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [term 1 LEADER]: Becoming Leader. State: Replica: de8ae17fbc2d46b189063274e602d4dc, State: Running, Role: LEADER
I20260812 05:15:46.162314 21698 sys_catalog.cc:564] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:46.162369 21703 consensus_queue.cc:237] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8ae17fbc2d46b189063274e602d4dc" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33461 } }
I20260812 05:15:46.162747 21703 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader de8ae17fbc2d46b189063274e602d4dc. Latest consensus state: current_term: 1 leader_uuid: "de8ae17fbc2d46b189063274e602d4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8ae17fbc2d46b189063274e602d4dc" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33461 } } }
I20260812 05:15:46.162842 21703 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:46.162758 21705 sys_catalog.cc:455] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de8ae17fbc2d46b189063274e602d4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de8ae17fbc2d46b189063274e602d4dc" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33461 } } }
I20260812 05:15:46.163026 21705 sys_catalog.cc:458] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc [sys.catalog]: This master's current role is: LEADER
W20260812 05:15:46.164335 21718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:15:46.164458 21718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:15:46.164865 21710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:46.165076 21710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:46.170748 21710 catalog_manager.cc:1347] Generated new cluster ID: c8405433092d45028871c5030a60bdd9
I20260812 05:15:46.170822 21710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:46.174742 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.157846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58464 (local address 127.18.166.254:33461)
0812 05:15:46.158087 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:46.158098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:46.158135 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:15:46.158753 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:46.158763 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:46.158838 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:46.159005 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:46.160045 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.162006 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.163263 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.164184 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.164304 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.170226 (+ 5922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:46.170233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:46.170233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:46.170253 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:46.172756 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.173316 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.173320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.173320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.173465 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.173710 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.173710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.173711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.174034 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:46.174190 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:46.174366 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 05:15:46.174519 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260812 05:15:46.181319 21710 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:46.182200 21710 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:46.188154 21710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de8ae17fbc2d46b189063274e602d4dc: Generated new TSK 0
I20260812 05:15:46.188429 21710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:46.221252 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:33461
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:46.311509 21724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:46.311842 21724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:46.311889 21724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:46.313231 21724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:46.313316 21724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:46.313366 21724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:46.315048 21724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:33461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:46.315436 21724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:46.315709 21724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:46.321305 21733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:46.321322 21731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:46.321348 21724 server_base.cc:1034] running on GCE node
W20260812 05:15:46.321309 21730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:46.321820 21724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:46.322132 21724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:46.323328 21724 hybrid_clock.cc:648] HybridClock initialized: now 1786511746323315 us; error 51 us; skew 500 ppm
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:46.326355 21724 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:46.326887 21724 webserver.cc:460] Webserver started at http://127.18.166.193:36169/ using document root <none> and password file <none>
I20260812 05:15:46.327167 21724 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:46.327270 21724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:46.327467 21724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:46.328617 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "02beac015e7a4e5cabc81173148e8f47"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "9732ebb49abfce42b106503766097c9d"
server_key_iv: "3a454375786fdd858f056f8f2433f7c6"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.329020 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "02beac015e7a4e5cabc81173148e8f47"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "9732ebb49abfce42b106503766097c9d"
server_key_iv: "3a454375786fdd858f056f8f2433f7c6"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.330713 21724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:15:46.331662 21740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.331914 21724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:46.332038 21724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "02beac015e7a4e5cabc81173148e8f47"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "9732ebb49abfce42b106503766097c9d"
server_key_iv: "3a454375786fdd858f056f8f2433f7c6"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.332155 21724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:46.352391 21724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:46.354846 21724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:46.355065 21724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:46.355341 21724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:46.355702 21724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:46.355767 21724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.355806 21724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:46.355835 21724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.362531 21724 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:45293
I20260812 05:15:46.362591 21857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:45293 every 8 connection(s)
I20260812 05:15:46.363006 21724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:46.368234 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 21724
I20260812 05:15:46.368352 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:46.368566 19099 external_mini_cluster.cc:1428] Setting key bd18c19eb095e4689b2c7a1d4c2356b7
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:46.374145 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.363913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:33675 (local address 127.18.166.254:33461)
0812 05:15:46.363990 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:46.363992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:46.364335 (+ 343us) server_negotiation.cc:407] Connection header received
0812 05:15:46.364877 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:46.364879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:46.364890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:46.364945 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:46.366122 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.366648 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.367729 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.367964 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.368023 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.370863 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:46.370865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:46.370865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:46.370875 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:46.372383 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.372912 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.372913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.372914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.372987 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.373297 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.373298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.373298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.373496 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:46.373596 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:46.373921 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 05:15:46.374043 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:15:46.374365 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.363899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:33461 (local address 127.18.166.193:33675)
0812 05:15:46.364168 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:46.364211 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:46.364772 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:46.365003 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:46.365007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:46.365791 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:46.366009 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:46.366014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.366753 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.366754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:46.367637 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:46.367639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.368084 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.368086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:46.368255 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.368924 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:46.368934 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:46.370735 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:46.372494 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:46.372505 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:46.372508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:46.372814 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:46.373066 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:46.373068 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:46.373068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:46.373208 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:46.373583 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:46.373592 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:46.373771 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 05:15:46.373900 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 05:15:46.374086 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":162,"threads_started":1}
I20260812 05:15:46.374985 21858 heartbeater.cc:347] Connected to a master server at 127.18.166.254:33461
I20260812 05:15:46.375124 21858 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:46.375381 21858 heartbeater.cc:511] Master 127.18.166.254:33461 requested a full tablet report, sending...
I20260812 05:15:46.376029 21660 ts_manager.cc:194] Registered new tserver with Master: 02beac015e7a4e5cabc81173148e8f47 (127.18.166.193:45293)
I20260812 05:15:46.376698 21660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:33675
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:46.401119 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:33461
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:46.487484 21864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:46.487730 21864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:46.487759 21864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:46.489025 21864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:46.489075 21864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:46.489104 21864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:46.490710 21864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:33461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:46.490949 21864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:46.491156 21864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:46.496677 21872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:46.496683 21870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:46.496954 21864 server_base.cc:1034] running on GCE node
W20260812 05:15:46.496680 21869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:46.497287 21864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:46.497632 21864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:46.498798 21864 hybrid_clock.cc:648] HybridClock initialized: now 1786511746498788 us; error 36 us; skew 500 ppm
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:46.501718 21864 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:46.502300 21864 webserver.cc:460] Webserver started at http://127.18.166.194:40115/ using document root <none> and password file <none>
I20260812 05:15:46.502580 21864 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:46.502656 21864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:46.502820 21864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:46.503960 21864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "81ce3fcb8ca54a2983430b7688fb9a0f"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "1caf81deb454b565a5fcb2a3222f1875"
server_key_iv: "8aee2472d8ca7ec9731c0b75ffef5188"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.504398 21864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "81ce3fcb8ca54a2983430b7688fb9a0f"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "1caf81deb454b565a5fcb2a3222f1875"
server_key_iv: "8aee2472d8ca7ec9731c0b75ffef5188"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.505968 21864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260812 05:15:46.506857 21880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.507050 21864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:46.507162 21864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "81ce3fcb8ca54a2983430b7688fb9a0f"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "1caf81deb454b565a5fcb2a3222f1875"
server_key_iv: "8aee2472d8ca7ec9731c0b75ffef5188"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.507266 21864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:46.518595 21864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:46.520926 21864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:46.521145 21864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:46.521482 21864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:46.521850 21864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:46.521909 21864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.521946 21864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:46.521974 21864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.528702 21864 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38003
I20260812 05:15:46.528760 21999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38003 every 8 connection(s)
I20260812 05:15:46.529127 21864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:46.537024 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 21864
I20260812 05:15:46.537176 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:46.537460 19099 external_mini_cluster.cc:1428] Setting key 3685abf49e7e9f4f8fd698890805325f
I20260812 05:15:46.540136 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.530020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:53957 (local address 127.18.166.254:33461)
0812 05:15:46.530089 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:46.530091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:46.530317 (+ 226us) server_negotiation.cc:407] Connection header received
0812 05:15:46.530830 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:46.530831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:46.530842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:46.530910 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:46.531959 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.532463 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.533611 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.533833 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.533912 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.536582 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:46.536584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:46.536585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:46.536594 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:46.538471 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.538975 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.538977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.538978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.539054 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.539333 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.539334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.539334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.539504 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:46.539584 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:46.539881 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 05:15:46.539994 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:15:46.540392 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.529965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:33461 (local address 127.18.166.194:53957)
0812 05:15:46.530166 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:46.530214 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:46.530735 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:46.530972 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:46.530975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:46.531615 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:46.531842 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:46.531848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.532587 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.532588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:46.533497 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:46.533500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.533948 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.533948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:46.534064 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.534685 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:46.534695 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:46.536450 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:46.538582 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:46.538595 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:46.538598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:46.538882 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:46.539132 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:46.539133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:46.539133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:46.539262 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:46.539577 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:46.539582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:46.539733 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 05:15:46.539826 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 05:15:46.540062 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260812 05:15:46.540966 22000 heartbeater.cc:347] Connected to a master server at 127.18.166.254:33461
I20260812 05:15:46.541109 22000 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:46.541342 22000 heartbeater.cc:511] Master 127.18.166.254:33461 requested a full tablet report, sending...
I20260812 05:15:46.541898 21660 ts_manager.cc:194] Registered new tserver with Master: 81ce3fcb8ca54a2983430b7688fb9a0f (127.18.166.194:38003)
I20260812 05:15:46.542323 21660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:53957
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:46.568868 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:33461
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:46.656774 22005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:46.657022 22005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:46.657051 22005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:46.658346 22005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:46.658399 22005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:46.658430 22005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:46.659956 22005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:33461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:46.660188 22005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:46.660401 22005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:46.666610 22014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:46.666610 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:46.666667 22012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:46.666821 22005 server_base.cc:1034] running on GCE node
I20260812 05:15:46.667057 22005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:46.667330 22005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:46.668509 22005 hybrid_clock.cc:648] HybridClock initialized: now 1786511746668496 us; error 47 us; skew 500 ppm
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:46.671327 22005 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:46.671793 22005 webserver.cc:460] Webserver started at http://127.18.166.195:39097/ using document root <none> and password file <none>
I20260812 05:15:46.671993 22005 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:46.672035 22005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:46.672133 22005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:46.673210 22005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "ba8edce7e9004154b5e8c719270ec8ef"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "8c2abd788feee85f99581a5fd20cbcf2"
server_key_iv: "579be57a4cc421a896dc7c02e5a2c2d4"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.673642 22005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "ba8edce7e9004154b5e8c719270ec8ef"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "8c2abd788feee85f99581a5fd20cbcf2"
server_key_iv: "579be57a4cc421a896dc7c02e5a2c2d4"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.675165 22005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:46.675958 22022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.676190 22005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:46.676297 22005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "ba8edce7e9004154b5e8c719270ec8ef"
format_stamp: "Formatted at 2026-08-12 05:15:46 on dist-test-slave-srgv"
server_key: "8c2abd788feee85f99581a5fd20cbcf2"
server_key_iv: "579be57a4cc421a896dc7c02e5a2c2d4"
server_key_version: "encryptionkey@0"
I20260812 05:15:46.676385 22005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:46.686905 22005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:46.689400 22005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:46.689688 22005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:46.689946 22005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:46.690311 22005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:46.690374 22005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.690419 22005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:46.690447 22005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:46.696714 22005 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:44795
I20260812 05:15:46.696796 22142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:44795 every 8 connection(s)
I20260812 05:15:46.697198 22005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[21613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511746, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:46.705166 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22005
I20260812 05:15:46.705277 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:46.705536 19099 external_mini_cluster.cc:1428] Setting key a6009752a5c4c275b3723075f82696d8
I20260812 05:15:46.707645 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.698147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39761 (local address 127.18.166.254:33461)
0812 05:15:46.698193 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:46.698195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:46.698426 (+ 231us) server_negotiation.cc:407] Connection header received
0812 05:15:46.698920 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:46.698922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:46.698933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:46.698987 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:46.699993 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.700499 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.701631 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.701824 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.701883 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.704381 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:46.704383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:46.704383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:46.704392 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:46.706149 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.706725 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.706726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.706726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.706799 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:46.707057 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:46.707057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:46.707058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:46.707175 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:46.707233 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:46.707502 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 05:15:46.707576 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:15:46.707899 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:46.698119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:33461 (local address 127.18.166.195:39761)
0812 05:15:46.698288 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:46.698327 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:46.698828 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:46.699046 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:46.699049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:46.699667 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:46.699896 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:46.699901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.700602 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.700603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:46.701546 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:46.701549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:46.701910 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:46.701911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:46.702021 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:46.702690 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:46.702699 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:46.704260 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:46.706295 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:46.706306 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:46.706309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:46.706624 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:46.706867 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:46.706868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:46.706868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:46.706993 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:46.707239 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:46.707243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:46.707378 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 05:15:46.707470 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 05:15:46.707613 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 05:15:46.708443 22143 heartbeater.cc:347] Connected to a master server at 127.18.166.254:33461
I20260812 05:15:46.708549 22143 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:46.708714 22143 heartbeater.cc:511] Master 127.18.166.254:33461 requested a full tablet report, sending...
I20260812 05:15:46.709143 21660 ts_manager.cc:194] Registered new tserver with Master: ba8edce7e9004154b5e8c719270ec8ef (127.18.166.195:44795)
I20260812 05:15:46.709583 21660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:39761
I20260812 05:15:46.717878 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 05:15:46.731191 21702 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:46.730141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.18.166.254:33461)
0812 05:15:46.730220 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:46.730223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:46.730329 (+ 106us) server_negotiation.cc:407] Connection header received
0812 05:15:46.730385 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:46.730386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:46.730397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:46.730450 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:46.731107 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40212: BlockingRecv error: recv got EOF from 127.0.0.1:40212 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:15:46.732064 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 21724
I20260812 05:15:46.738732 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 21864
I20260812 05:15:46.744768 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22005
I20260812 05:15:46.751257 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 21626
2026-08-12T05:15:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1963 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[22157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[22157](info): set up 2 sockets
Aug 12 05:15:46 dist-test-slave-srgv krb5kdc[22157](info): commencing operation
krb5kdc: starting...
W20260812 05:15:47.939735 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 05:15:48.177878 19099 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.238s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 05:15:48.411181 19099 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.233s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 05:15:48.646396 19099 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.235s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/test-user.keytab.
W20260812 05:15:48.761430 19099 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.115s user 0.001s sys 0.001s
Aug 12 05:15:48 dist-test-slave-srgv krb5kdc[22157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:48.810897 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43787
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:43787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:48.899271 22173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:48.899577 22173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:48.899623 22173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:48.900955 22173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:48.901058 22173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:48.901093 22173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:48.901114 22173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:48.901142 22173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:48.903002 22173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:43787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:48.903380 22173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:48.903658 22173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:48.909344 22182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:48.909541 22173 server_base.cc:1034] running on GCE node
W20260812 05:15:48.909333 22180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:48.909340 22179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:48.909947 22173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:48.910315 22173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:48.911501 22173 hybrid_clock.cc:648] HybridClock initialized: now 1786511748911490 us; error 34 us; skew 500 ppm
Aug 12 05:15:48 dist-test-slave-srgv krb5kdc[22157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511748, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:48.914613 22173 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:48.915158 22173 webserver.cc:460] Webserver started at http://127.18.166.254:40569/ using document root <none> and password file <none>
I20260812 05:15:48.915460 22173 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:48.915546 22173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:48.915726 22173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:48.916965 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "32e7e354f1a24b57ab71beece80da435"
format_stamp: "Formatted at 2026-08-12 05:15:48 on dist-test-slave-srgv"
server_key: "14a6cb367b2f0620e46f39f9edd9a19b"
server_key_iv: "172ae71d91ecc459e9dac89075f77b7f"
server_key_version: "encryptionkey@0"
I20260812 05:15:48.917392 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "32e7e354f1a24b57ab71beece80da435"
format_stamp: "Formatted at 2026-08-12 05:15:48 on dist-test-slave-srgv"
server_key: "14a6cb367b2f0620e46f39f9edd9a19b"
server_key_iv: "172ae71d91ecc459e9dac89075f77b7f"
server_key_version: "encryptionkey@0"
I20260812 05:15:48.919077 22173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:48.920010 22189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:48.920269 22173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:48.920392 22173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "32e7e354f1a24b57ab71beece80da435"
format_stamp: "Formatted at 2026-08-12 05:15:48 on dist-test-slave-srgv"
server_key: "14a6cb367b2f0620e46f39f9edd9a19b"
server_key_iv: "172ae71d91ecc459e9dac89075f77b7f"
server_key_version: "encryptionkey@0"
I20260812 05:15:48.920483 22173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:48.933493 22173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:48.936028 22173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:48.936251 22173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:48.940929 22173 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:43787
I20260812 05:15:48.940987 22244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:43787 every 8 connection(s)
I20260812 05:15:48.941745 22173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:48.942565 22245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:48.945588 22245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435: Bootstrap starting.
I20260812 05:15:48.946346 22245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:48.946722 22245 log.cc:824] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:48.946993 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22173
I20260812 05:15:48.947131 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:48.947319 19099 external_mini_cluster.cc:1428] Setting key 3e8ce11c51052c0ace4513d3c7f38bb1
I20260812 05:15:48.947953 22245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435: No bootstrap required, opened a new log
I20260812 05:15:48.951170 22245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e7e354f1a24b57ab71beece80da435" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43787 } }
I20260812 05:15:48.951411 22245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:48.951449 22245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e7e354f1a24b57ab71beece80da435, State: Initialized, Role: FOLLOWER
I20260812 05:15:48.951592 22245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e7e354f1a24b57ab71beece80da435" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43787 } }
I20260812 05:15:48.951678 22245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:48.951709 22245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:48.951756 22245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:48.952582 22245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e7e354f1a24b57ab71beece80da435" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43787 } }
I20260812 05:15:48.952713 22245 leader_election.cc:302] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32e7e354f1a24b57ab71beece80da435; no voters:
I20260812 05:15:48.953034 22245 leader_election.cc:288] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:48.953066 22250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:48.953209 22250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [term 1 LEADER]: Becoming Leader. State: Replica: 32e7e354f1a24b57ab71beece80da435, State: Running, Role: LEADER
I20260812 05:15:48.953305 22250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e7e354f1a24b57ab71beece80da435" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43787 } }
I20260812 05:15:48.953372 22245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:48.953717 22251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32e7e354f1a24b57ab71beece80da435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e7e354f1a24b57ab71beece80da435" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43787 } } }
I20260812 05:15:48.954027 22251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:48.953938 22252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32e7e354f1a24b57ab71beece80da435. Latest consensus state: current_term: 1 leader_uuid: "32e7e354f1a24b57ab71beece80da435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32e7e354f1a24b57ab71beece80da435" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43787 } } }
I20260812 05:15:48.954367 22252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:48.954540 22259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:48.955762 22259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:15:48 dist-test-slave-srgv krb5kdc[22157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:48.957806 22259 catalog_manager.cc:1347] Generated new cluster ID: 5802bc9b66c24e68841177f290c3f53d
I20260812 05:15:48.957878 22259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:48.961818 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:48.947951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.18.166.254:43787)
0812 05:15:48.948203 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:48.948214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:48.948259 (+ 45us) server_negotiation.cc:407] Connection header received
0812 05:15:48.949566 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:48.949578 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:48.949662 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:48.949854 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:48.950577 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:48.952008 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:48.952968 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:48.953761 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:48.954289 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:48.957210 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:48.957215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:48.957215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:48.957231 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:48.959651 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:48.960212 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:48.960217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:48.960217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:48.960378 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:48.960637 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:48.960638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:48.960639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:48.961033 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:48.961225 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:48.961340 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 05:15:48.961542 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260812 05:15:48.963462 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:15:48.968720 22259 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:48.969342 22259 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:48.973017 22259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32e7e354f1a24b57ab71beece80da435: Generated new TSK 0
I20260812 05:15:48.973250 22259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:15:48.984047 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:48.978940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.18.166.254:43787)
0812 05:15:48.979008 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:48.979011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:48.979169 (+ 158us) server_negotiation.cc:407] Connection header received
0812 05:15:48.979214 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:48.979215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:48.979228 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:48.979318 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:48.979783 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:48.980635 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:48.982030 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:48.982408 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:48.982503 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:48.983409 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:48.983411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:48.983411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:48.983700 (+ 289us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:48.983723 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:48.983731 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:48.983808 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:48.983908 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 05:15:48.983950 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:15:48.987735 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22173
2026-08-12T05:15:48Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2234 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:49 dist-test-slave-srgv krb5kdc[22278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:49 dist-test-slave-srgv krb5kdc[22278](info): set up 2 sockets
Aug 12 05:15:49 dist-test-slave-srgv krb5kdc[22278](info): commencing operation
krb5kdc: starting...
W20260812 05:15:50.080598 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:50.224784 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36261
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:36261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:50.310428 22292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:50.310690 22292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:50.310719 22292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:50.312088 22292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:50.312153 22292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:50.312168 22292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:50.312183 22292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:50.312196 22292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:50.314028 22292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:36261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:50.314379 22292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:50.314663 22292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:50.320346 22301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.320403 22299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.320358 22298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:50.320756 22292 server_base.cc:1034] running on GCE node
I20260812 05:15:50.321086 22292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:50.321518 22292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:50.322741 22292 hybrid_clock.cc:648] HybridClock initialized: now 1786511750322710 us; error 43 us; skew 500 ppm
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:50.325834 22292 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:50.326464 22292 webserver.cc:460] Webserver started at http://127.18.166.254:43735/ using document root <none> and password file <none>
I20260812 05:15:50.326742 22292 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:50.326831 22292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:50.327010 22292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:50.328265 22292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "888d807f65504a8b9389bb3df2e6ba12"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "14975005573fd7189f34fbf371c975be"
server_key_iv: "f83fd358e48233eb7e6314613559681c"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.328706 22292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "888d807f65504a8b9389bb3df2e6ba12"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "14975005573fd7189f34fbf371c975be"
server_key_iv: "f83fd358e48233eb7e6314613559681c"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.330476 22292 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:15:50.331413 22309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.331624 22292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:50.331749 22292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "888d807f65504a8b9389bb3df2e6ba12"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "14975005573fd7189f34fbf371c975be"
server_key_iv: "f83fd358e48233eb7e6314613559681c"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.331843 22292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:50.348031 22292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:50.351032 22292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:50.351245 22292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:50.356437 22292 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:36261
I20260812 05:15:50.356645 22364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:36261 every 8 connection(s)
I20260812 05:15:50.356976 22292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:50.358500 22365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:50.361084 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22292
I20260812 05:15:50.361189 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:50.361356 22365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12: Bootstrap starting.
I20260812 05:15:50.361415 19099 external_mini_cluster.cc:1428] Setting key 3ebd7a2f7d15fd32b51ed1d95be35f94
I20260812 05:15:50.362349 22365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:50.362754 22365 log.cc:824] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:50.363719 22365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12: No bootstrap required, opened a new log
I20260812 05:15:50.366634 22365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d807f65504a8b9389bb3df2e6ba12" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36261 } }
I20260812 05:15:50.366847 22365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:50.366874 22365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888d807f65504a8b9389bb3df2e6ba12, State: Initialized, Role: FOLLOWER
I20260812 05:15:50.367044 22365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d807f65504a8b9389bb3df2e6ba12" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36261 } }
I20260812 05:15:50.367146 22365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:50.367199 22365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:50.367270 22365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:50.368273 22365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d807f65504a8b9389bb3df2e6ba12" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36261 } }
I20260812 05:15:50.368418 22365 leader_election.cc:302] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 888d807f65504a8b9389bb3df2e6ba12; no voters:
I20260812 05:15:50.368633 22365 leader_election.cc:288] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:50.368708 22370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:50.368937 22370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [term 1 LEADER]: Becoming Leader. State: Replica: 888d807f65504a8b9389bb3df2e6ba12, State: Running, Role: LEADER
I20260812 05:15:50.368985 22365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:50.369060 22370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d807f65504a8b9389bb3df2e6ba12" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36261 } }
I20260812 05:15:50.369563 22372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 888d807f65504a8b9389bb3df2e6ba12. Latest consensus state: current_term: 1 leader_uuid: "888d807f65504a8b9389bb3df2e6ba12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d807f65504a8b9389bb3df2e6ba12" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36261 } } }
I20260812 05:15:50.369575 22371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "888d807f65504a8b9389bb3df2e6ba12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d807f65504a8b9389bb3df2e6ba12" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36261 } } }
I20260812 05:15:50.369800 22371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:50.370039 22372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:50.370249 22379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:50.370977 22379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:50.374428 22379 catalog_manager.cc:1347] Generated new cluster ID: 962617d737aa4f9c8b28a1dc0812fab2
I20260812 05:15:50.374527 22379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:50.376371 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.362115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.18.166.254:36261)
0812 05:15:50.362372 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:50.362383 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:50.362426 (+ 43us) server_negotiation.cc:407] Connection header received
0812 05:15:50.363191 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:50.363200 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:50.363264 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:50.363488 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:50.364651 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.366360 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.367557 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.368257 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.368405 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.372166 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:50.372171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:50.372172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:50.372188 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:50.374271 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.374845 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.374849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.374850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.374977 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.375246 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.375247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.375247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.375600 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:50.375775 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:50.375936 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 05:15:50.376103 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
I20260812 05:15:50.379539 22379 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:50.380153 22379 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:50.392170 22379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 888d807f65504a8b9389bb3df2e6ba12: Generated new TSK 0
I20260812 05:15:50.392470 22379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:50.432732 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36261
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:50.524206 22392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:50.524506 22392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:50.524557 22392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:50.526048 22392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:50.526156 22392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:50.526221 22392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:50.528036 22392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:50.528394 22392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:50.528673 22392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:50.534269 22398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.534269 22400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.534282 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:50.534361 22392 server_base.cc:1034] running on GCE node
I20260812 05:15:50.534801 22392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:50.535112 22392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:50.536368 22392 hybrid_clock.cc:648] HybridClock initialized: now 1786511750536345 us; error 60 us; skew 500 ppm
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:50.539474 22392 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:50.540046 22392 webserver.cc:460] Webserver started at http://127.18.166.193:37409/ using document root <none> and password file <none>
I20260812 05:15:50.540349 22392 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:50.540401 22392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:50.540580 22392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:50.541910 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "e7f8a1fbf46e428b8e9f16b0eb27bfcb"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "8f39a198e4901fb503a851485195d89a"
server_key_iv: "3d3639c2071d5ec37e6256fc02d32af3"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.542342 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "e7f8a1fbf46e428b8e9f16b0eb27bfcb"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "8f39a198e4901fb503a851485195d89a"
server_key_iv: "3d3639c2071d5ec37e6256fc02d32af3"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.543937 22392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:50.544898 22407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.545166 22392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:50.545261 22392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "e7f8a1fbf46e428b8e9f16b0eb27bfcb"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "8f39a198e4901fb503a851485195d89a"
server_key_iv: "3d3639c2071d5ec37e6256fc02d32af3"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.545372 22392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:50.556834 22392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:50.559415 22392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:50.559659 22392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:50.559973 22392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:50.560354 22392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:50.560423 22392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.560461 22392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:50.560518 22392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.568898 22392 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:33747
I20260812 05:15:50.568944 22528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:33747 every 8 connection(s)
I20260812 05:15:50.569347 22392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:50.570014 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22392
I20260812 05:15:50.570150 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:50.570381 19099 external_mini_cluster.cc:1428] Setting key a5138bb2ceba359f29827b627bbff2b0
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:15:50.582705 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.571070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:43189 (local address 127.18.166.254:36261)
0812 05:15:50.571121 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:50.571124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:50.571439 (+ 315us) server_negotiation.cc:407] Connection header received
0812 05:15:50.571971 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:50.571972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:50.571987 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:50.572068 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:50.573198 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.573832 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.575593 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.575983 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.576039 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.579196 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:50.579198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:50.579198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:50.579210 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:50.581005 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.581622 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.581623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.581624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.581691 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.581958 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.581959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.581959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.582083 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:50.582222 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:50.582524 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 05:15:50.582628 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:15:50.582927 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.571074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36261 (local address 127.18.166.193:43189)
0812 05:15:50.571267 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:50.571334 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:50.571858 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:50.572129 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:50.572133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:50.572811 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:50.573036 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:50.573042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.573955 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.573957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:50.575447 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:50.575452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.576171 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.576172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:50.576368 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.577053 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:50.577067 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:50.579040 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:50.581147 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:50.581160 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:50.581163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:50.581533 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:50.581767 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:50.581768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:50.581768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:50.581892 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:50.582156 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:50.582161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:50.582369 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 05:15:50.582488 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 05:15:50.582662 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260812 05:15:50.583635 22529 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36261
I20260812 05:15:50.583781 22529 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
I20260812 05:15:50.584045 22529 heartbeater.cc:511] Master 127.18.166.254:36261 requested a full tablet report, sending...
I20260812 05:15:50.584687 22326 ts_manager.cc:194] Registered new tserver with Master: e7f8a1fbf46e428b8e9f16b0eb27bfcb (127.18.166.193:33747)
I20260812 05:15:50.585381 22326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:43189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:50.601109 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36261
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:50.689532 22535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:50.689795 22535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:50.689826 22535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:50.691126 22535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:50.691183 22535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:50.691212 22535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:50.692857 22535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:50.693106 22535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:50.693318 22535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:50.698863 22540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:50.699084 22535 server_base.cc:1034] running on GCE node
W20260812 05:15:50.698864 22541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.698863 22543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:50.699419 22535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:50.699735 22535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:50.700932 22535 hybrid_clock.cc:648] HybridClock initialized: now 1786511750700926 us; error 42 us; skew 500 ppm
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:50.703859 22535 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:50.704414 22535 webserver.cc:460] Webserver started at http://127.18.166.194:38877/ using document root <none> and password file <none>
I20260812 05:15:50.704707 22535 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:50.704787 22535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:50.705010 22535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:50.706277 22535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "d24eb494f18d46b9a3fec4bc5ba49b4d"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "cbd2571d9d01b2e61da0f49e8ca157c4"
server_key_iv: "a6b1c93cf8af9f17db7800fdd455fb5a"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.706705 22535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "d24eb494f18d46b9a3fec4bc5ba49b4d"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "cbd2571d9d01b2e61da0f49e8ca157c4"
server_key_iv: "a6b1c93cf8af9f17db7800fdd455fb5a"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.708272 22535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:15:50.709180 22550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.709388 22535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:50.709559 22535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "d24eb494f18d46b9a3fec4bc5ba49b4d"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "cbd2571d9d01b2e61da0f49e8ca157c4"
server_key_iv: "a6b1c93cf8af9f17db7800fdd455fb5a"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.709673 22535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:50.723331 22535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:50.725693 22535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:50.725906 22535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:50.726190 22535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:50.726552 22535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:50.726612 22535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.726680 22535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:50.726712 22535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.733407 22535 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41307
I20260812 05:15:50.733484 22670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41307 every 8 connection(s)
I20260812 05:15:50.733942 22535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:50.737244 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22535
I20260812 05:15:50.737377 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:50.737625 19099 external_mini_cluster.cc:1428] Setting key e1f87d37b72b98cc378adeb4a68b7dee
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:50.745784 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.734799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:36747 (local address 127.18.166.254:36261)
0812 05:15:50.734881 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:50.734884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:50.735151 (+ 267us) server_negotiation.cc:407] Connection header received
0812 05:15:50.735757 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:50.735758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:50.735772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:50.735832 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:50.736978 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.737784 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.739404 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.739610 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.739682 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.742465 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:50.742467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:50.742467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:50.742477 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:50.744076 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.744605 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.744607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.744607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.744679 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.744970 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.744971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.744972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.745110 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:50.745204 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:50.745577 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 05:15:50.745678 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:15:50.746031 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.734771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36261 (local address 127.18.166.194:36747)
0812 05:15:50.734989 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:50.735033 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:50.735632 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:50.735947 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:50.735951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:50.736624 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:50.736843 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:50.736848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.737892 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.737894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:50.739283 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:50.739287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.739724 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.739725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:50.739899 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.740625 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:50.740635 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:50.742322 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:50.744214 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:50.744228 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:50.744231 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:50.744512 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:50.744766 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:50.744767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:50.744767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:50.744895 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:50.745193 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:50.745201 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:50.745367 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 05:15:50.745551 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 05:15:50.745753 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":114,"threads_started":1}
I20260812 05:15:50.746594 22671 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36261
I20260812 05:15:50.746696 22671 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:50.746906 22671 heartbeater.cc:511] Master 127.18.166.254:36261 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:15:50.747396 22326 ts_manager.cc:194] Registered new tserver with Master: d24eb494f18d46b9a3fec4bc5ba49b4d (127.18.166.194:41307)
I20260812 05:15:50.747805 22326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:36747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:50.766778 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36261
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:50.857227 22676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:50.857496 22676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:50.857525 22676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:50.858758 22676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:50.858803 22676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:50.858830 22676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:50.860355 22676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:50.860581 22676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:50.860790 22676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:50.866462 22682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.866462 22684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:50.866565 22681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:50.866792 22676 server_base.cc:1034] running on GCE node
I20260812 05:15:50.867046 22676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:50.867300 22676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:50.868455 22676 hybrid_clock.cc:648] HybridClock initialized: now 1786511750868442 us; error 34 us; skew 500 ppm
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:50.871701 22676 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:50.872180 22676 webserver.cc:460] Webserver started at http://127.18.166.195:45383/ using document root <none> and password file <none>
I20260812 05:15:50.872381 22676 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:50.872424 22676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:50.872527 22676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:50.873793 22676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "edbdeb98160b425d8e87330a5b6e0e94"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "d05b8f96ba52d96e73514015493438bc"
server_key_iv: "9d0d3dcd641e25ee7401aab5c5f09df4"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.874202 22676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "edbdeb98160b425d8e87330a5b6e0e94"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "d05b8f96ba52d96e73514015493438bc"
server_key_iv: "9d0d3dcd641e25ee7401aab5c5f09df4"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.875700 22676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:50.876611 22691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.876806 22676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:50.876921 22676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "edbdeb98160b425d8e87330a5b6e0e94"
format_stamp: "Formatted at 2026-08-12 05:15:50 on dist-test-slave-srgv"
server_key: "d05b8f96ba52d96e73514015493438bc"
server_key_iv: "9d0d3dcd641e25ee7401aab5c5f09df4"
server_key_version: "encryptionkey@0"
I20260812 05:15:50.877012 22676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:50.898139 22676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:50.900344 22676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:50.900547 22676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:50.900813 22676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:50.901163 22676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:50.901225 22676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.901271 22676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:50.901297 22676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:50.907956 22676 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:40071
I20260812 05:15:50.908001 22810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:40071 every 8 connection(s)
I20260812 05:15:50.908397 22676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:50.913352 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22676
I20260812 05:15:50.913519 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:50.913751 19099 external_mini_cluster.cc:1428] Setting key fa71a5bc9078f344597b6a3f631e1296
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:50.918919 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.909255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39373 (local address 127.18.166.254:36261)
0812 05:15:50.909330 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:50.909333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:50.909654 (+ 321us) server_negotiation.cc:407] Connection header received
0812 05:15:50.910162 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:50.910163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:50.910174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:50.910231 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:50.911264 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.911786 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.912908 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.913138 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.913229 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.916082 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:50.916084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:50.916084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:50.916093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:50.917582 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.918047 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.918048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.918048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.918107 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.918341 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.918341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.918341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.918458 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:50.918513 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:50.918766 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 05:15:50.918846 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:15:50.919190 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.909241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36261 (local address 127.18.166.195:39373)
0812 05:15:50.909503 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:50.909555 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:50.910063 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:50.910299 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:50.910303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:50.910938 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:50.911147 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:50.911152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.911890 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.911891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:50.912762 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:50.912764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.913251 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.913252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:50.913476 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.914215 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:50.914227 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:50.915973 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:50.917673 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:50.917683 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:50.917686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:50.917967 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:50.918164 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:50.918164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:50.918165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:50.918275 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:50.918501 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:50.918504 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:50.918636 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 05:15:50.918735 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 05:15:50.918886 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":117,"threads_started":1}
I20260812 05:15:50.919840 22811 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36261
I20260812 05:15:50.919963 22811 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:50.920137 22811 heartbeater.cc:511] Master 127.18.166.254:36261 requested a full tablet report, sending...
I20260812 05:15:50.920574 22326 ts_manager.cc:194] Registered new tserver with Master: edbdeb98160b425d8e87330a5b6e0e94 (127.18.166.195:40071)
I20260812 05:15:50.921003 22326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:39373
I20260812 05:15:50.926013 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 05:15:50 dist-test-slave-srgv krb5kdc[22278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511750, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:50.951960 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:50.944319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.18.166.254:36261)
0812 05:15:50.944376 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:50.944379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:50.944443 (+ 64us) server_negotiation.cc:407] Connection header received
0812 05:15:50.944512 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:50.944513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:50.944524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:50.944575 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:50.945235 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.945821 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.946582 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:50.946771 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:50.946835 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:50.949477 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:50.949479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:50.949479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:50.949487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:50.950877 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.951237 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.951237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.951238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.951291 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:50.951469 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:50.951470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:50.951470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:50.951631 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:50.951713 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:50.951819 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 05:15:50.951887 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 05:15:50.952349 22326 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48536
I20260812 05:15:50.953080 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22392
I20260812 05:15:50.958801 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22535
I20260812 05:15:50.964366 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22676
I20260812 05:15:50.972097 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22292
2026-08-12T05:15:50Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1986 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:51 dist-test-slave-srgv krb5kdc[22823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:51 dist-test-slave-srgv krb5kdc[22823](info): set up 2 sockets
Aug 12 05:15:51 dist-test-slave-srgv krb5kdc[22823](info): commencing operation
krb5kdc: starting...
W20260812 05:15:52.064317 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:52.197708 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43935
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:52.282996 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:52.283229 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:52.283255 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:52.284603 22837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:52.284665 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:52.284682 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:52.284694 22837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:52.284755 22837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:52.286480 22837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:43935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:52.286773 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:52.287098 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:52.292457 22845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:52.292685 22837 server_base.cc:1034] running on GCE node
W20260812 05:15:52.292476 22843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.292624 22842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:52.293076 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:52.293486 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:52.294690 22837 hybrid_clock.cc:648] HybridClock initialized: now 1786511752294679 us; error 44 us; skew 500 ppm
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:52.302241 22837 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:52.302824 22837 webserver.cc:460] Webserver started at http://127.18.166.254:45477/ using document root <none> and password file <none>
I20260812 05:15:52.303104 22837 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:52.303181 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:52.303354 22837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:52.304559 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "a0af3ff299ef43a387f97ddddfd930d0"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "571e7e9ae200e4bbd8c6ce12735a697e"
server_key_iv: "e927ca8891979280a59a4d7f73fe1f71"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.304970 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "a0af3ff299ef43a387f97ddddfd930d0"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "571e7e9ae200e4bbd8c6ce12735a697e"
server_key_iv: "e927ca8891979280a59a4d7f73fe1f71"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.306629 22837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:15:52.307530 22852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.307770 22837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:52.307893 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "a0af3ff299ef43a387f97ddddfd930d0"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "571e7e9ae200e4bbd8c6ce12735a697e"
server_key_iv: "e927ca8891979280a59a4d7f73fe1f71"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.307977 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:52.318768 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:52.320936 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:52.321141 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:52.325747 22837 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:43935
I20260812 05:15:52.325793 22908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:43935 every 8 connection(s)
I20260812 05:15:52.326364 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:52.327140 22909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:52.329679 22909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0: Bootstrap starting.
I20260812 05:15:52.330364 22909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:52.330704 22909 log.cc:824] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:52.331486 22909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0: No bootstrap required, opened a new log
I20260812 05:15:52.333202 22909 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0af3ff299ef43a387f97ddddfd930d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43935 } }
I20260812 05:15:52.333356 22909 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:52.333379 22909 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0af3ff299ef43a387f97ddddfd930d0, State: Initialized, Role: FOLLOWER
I20260812 05:15:52.333613 22909 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0af3ff299ef43a387f97ddddfd930d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43935 } }
I20260812 05:15:52.333689 22909 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:52.333735 22909 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:52.333788 22909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:52.333747 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22837
I20260812 05:15:52.333868 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:52.334049 19099 external_mini_cluster.cc:1428] Setting key 7d3454b0c82ace91f2ece43859704354
I20260812 05:15:52.335218 22909 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0af3ff299ef43a387f97ddddfd930d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43935 } }
I20260812 05:15:52.335348 22909 leader_election.cc:302] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0af3ff299ef43a387f97ddddfd930d0; no voters:
I20260812 05:15:52.335520 22909 leader_election.cc:288] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:52.335561 22914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:52.335673 22914 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [term 1 LEADER]: Becoming Leader. State: Replica: a0af3ff299ef43a387f97ddddfd930d0, State: Running, Role: LEADER
I20260812 05:15:52.335760 22914 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0af3ff299ef43a387f97ddddfd930d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43935 } }
I20260812 05:15:52.335824 22909 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:52.336136 22915 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0af3ff299ef43a387f97ddddfd930d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0af3ff299ef43a387f97ddddfd930d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43935 } } }
I20260812 05:15:52.336251 22915 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:52.336153 22916 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0af3ff299ef43a387f97ddddfd930d0. Latest consensus state: current_term: 1 leader_uuid: "a0af3ff299ef43a387f97ddddfd930d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0af3ff299ef43a387f97ddddfd930d0" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43935 } } }
I20260812 05:15:52.336627 22916 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:52.336910 22920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:52.337487 22920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:52.339273 22920 catalog_manager.cc:1347] Generated new cluster ID: 6ecc755f3a3241e4a9328f286491ce0c
I20260812 05:15:52.339336 22920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:52.345834 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.334644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.18.166.254:43935)
0812 05:15:52.334906 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:52.334918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:52.334957 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:15:52.335102 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:52.335110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:52.335172 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:52.335338 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:52.336195 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.337659 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.338795 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.339354 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.339463 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.342178 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:52.342182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:52.342182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:52.342194 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:52.344071 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.344502 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.344505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.344506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.344627 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.344862 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.344863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.344863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.345127 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:52.345317 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:52.345460 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 05:15:52.345615 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":130,"threads_started":1}
I20260812 05:15:52.356688 22920 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:52.357306 22920 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:52.364373 22920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0af3ff299ef43a387f97ddddfd930d0: Generated new TSK 0
I20260812 05:15:52.364638 22920 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:52.396586 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43935
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:52.484623 22936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:52.484907 22936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:52.484953 22936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:52.486287 22936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:52.486374 22936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:52.486425 22936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:52.488302 22936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:52.488642 22936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:52.488909 22936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:52.494465 22941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.494458 22942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.494458 22944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:52.494788 22936 server_base.cc:1034] running on GCE node
I20260812 05:15:52.495010 22936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:52.495311 22936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:52.496464 22936 hybrid_clock.cc:648] HybridClock initialized: now 1786511752496451 us; error 29 us; skew 500 ppm
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:52.499272 22936 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:52.499781 22936 webserver.cc:460] Webserver started at http://127.18.166.193:38251/ using document root <none> and password file <none>
I20260812 05:15:52.500003 22936 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:52.500046 22936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:52.500150 22936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:52.501277 22936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "eca90b40ad9042fc893854c9f54027e9"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "ff5d5c5b87728ccd819d7fecbb41cef1"
server_key_iv: "40816855b606091c6c50c650069d0f94"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.501714 22936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "eca90b40ad9042fc893854c9f54027e9"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "ff5d5c5b87728ccd819d7fecbb41cef1"
server_key_iv: "40816855b606091c6c50c650069d0f94"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.503123 22936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:52.503906 22951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.504096 22936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:52.504208 22936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "eca90b40ad9042fc893854c9f54027e9"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "ff5d5c5b87728ccd819d7fecbb41cef1"
server_key_iv: "40816855b606091c6c50c650069d0f94"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.504273 22936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:52.516857 22936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:52.519011 22936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:52.519137 22936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:52.519368 22936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:52.519717 22936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:52.519752 22936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.519774 22936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:52.519826 22936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.526163 22936 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:33585
I20260812 05:15:52.526266 23069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:33585 every 8 connection(s)
I20260812 05:15:52.526624 22936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:52.532924 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 22936
I20260812 05:15:52.533046 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:52.533298 19099 external_mini_cluster.cc:1428] Setting key d5777671ad58a6e7abb755c6916be4db
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:52.538071 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.527428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:34781 (local address 127.18.166.254:43935)
0812 05:15:52.527503 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:52.527505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:52.527742 (+ 237us) server_negotiation.cc:407] Connection header received
0812 05:15:52.528777 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:52.528778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:52.528798 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:52.528849 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:52.529604 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.530125 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.531195 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.531412 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.531507 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.534279 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:52.534281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:52.534282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:52.534292 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:52.536140 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.536732 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.536734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.536734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.536804 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.537145 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.537146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.537147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.537323 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:52.537437 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:52.537808 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 05:15:52.537922 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:15:52.538105 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.527395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43935 (local address 127.18.166.193:34781)
0812 05:15:52.527599 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:52.527638 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:52.528629 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:52.528940 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:52.528944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:52.529225 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:52.529500 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:52.529506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.530244 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.530245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:52.531079 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:52.531081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.531524 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.531525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:52.531682 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.532271 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:52.532280 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:52.534172 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:52.536274 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:52.536283 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:52.536286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:52.536565 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:52.536913 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:52.536914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:52.536914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:52.537061 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:52.537492 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:52.537505 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:52.537655 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 05:15:52.537766 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 05:15:52.537922 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260812 05:15:52.538769 23070 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43935
I20260812 05:15:52.538897 23070 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:52.539124 23070 heartbeater.cc:511] Master 127.18.166.254:43935 requested a full tablet report, sending...
I20260812 05:15:52.539697 22870 ts_manager.cc:194] Registered new tserver with Master: eca90b40ad9042fc893854c9f54027e9 (127.18.166.193:33585)
I20260812 05:15:52.540356 22870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:34781
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:52.562741 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43935
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:52.650533 23076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:52.650830 23076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:52.650882 23076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:52.652240 23076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:52.652334 23076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:52.652383 23076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:52.654136 23076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:52.654464 23076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:52.654731 23076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:52.660333 23081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.660337 23082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.660378 23084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:52.660370 23076 server_base.cc:1034] running on GCE node
I20260812 05:15:52.660920 23076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:52.661201 23076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:52.662380 23076 hybrid_clock.cc:648] HybridClock initialized: now 1786511752662370 us; error 33 us; skew 500 ppm
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:52.665258 23076 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:52.665818 23076 webserver.cc:460] Webserver started at http://127.18.166.194:32851/ using document root <none> and password file <none>
I20260812 05:15:52.666074 23076 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:52.666152 23076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:52.666317 23076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:52.667438 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "32d8b73cc5434913b5e8f7008f73995b"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "d6e45e4f2579c9763a89eebdb456b3a5"
server_key_iv: "0dafaeee6d15d5784ec74bdcad99bdb3"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.667835 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "32d8b73cc5434913b5e8f7008f73995b"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "d6e45e4f2579c9763a89eebdb456b3a5"
server_key_iv: "0dafaeee6d15d5784ec74bdcad99bdb3"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.669381 23076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:15:52.670285 23092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.670483 23076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:52.670598 23076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "32d8b73cc5434913b5e8f7008f73995b"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "d6e45e4f2579c9763a89eebdb456b3a5"
server_key_iv: "0dafaeee6d15d5784ec74bdcad99bdb3"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.670692 23076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:52.682874 23076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:52.685177 23076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:52.685377 23076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:52.685681 23076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:52.686033 23076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:52.686091 23076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.686139 23076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:52.686164 23076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.692770 23076 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:39753
I20260812 05:15:52.692827 23209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:39753 every 8 connection(s)
I20260812 05:15:52.693190 23076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:52.698758 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23076
I20260812 05:15:52.698907 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:52.699106 19099 external_mini_cluster.cc:1428] Setting key fcce74650f53e35c10a3c4979e7c998f
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:52.704010 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.694051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:47847 (local address 127.18.166.254:43935)
0812 05:15:52.694127 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:52.694130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:52.694340 (+ 210us) server_negotiation.cc:407] Connection header received
0812 05:15:52.695357 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:52.695359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:52.695373 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:52.695435 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:52.696143 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.696675 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.697750 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.697967 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.698051 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.700780 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:52.700782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:52.700782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:52.700791 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:52.702416 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.702935 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.702936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.702937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.703005 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.703287 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.703288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.703289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.703421 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:52.703498 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:52.703815 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 05:15:52.703915 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:15:52.704241 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.694029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43935 (local address 127.18.166.194:47847)
0812 05:15:52.694202 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:52.694242 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:52.695225 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:52.695517 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:52.695521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:52.695822 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:52.696027 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:52.696032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.696790 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.696792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:52.697632 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:52.697635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.698075 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.698076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:52.698207 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.698758 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:52.698769 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:52.700665 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:52.702546 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:52.702557 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:52.702559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:52.702832 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:52.703091 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:52.703091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:52.703092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:52.703217 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:52.703500 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:52.703509 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:52.703668 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 05:15:52.703781 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 05:15:52.703959 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260812 05:15:52.704780 23210 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43935
I20260812 05:15:52.704886 23210 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:52.705106 23210 heartbeater.cc:511] Master 127.18.166.254:43935 requested a full tablet report, sending...
I20260812 05:15:52.705610 22870 ts_manager.cc:194] Registered new tserver with Master: 32d8b73cc5434913b5e8f7008f73995b (127.18.166.194:39753)
I20260812 05:15:52.706194 22870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:47847
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:52.728207 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43935
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:52.815265 23215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:52.815495 23215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:52.815523 23215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:52.816700 23215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:52.816741 23215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:52.816768 23215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:52.818421 23215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:52.818646 23215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:52.818841 23215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:52.824034 23221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.824021 23223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:52.824044 23220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:52.824409 23215 server_base.cc:1034] running on GCE node
I20260812 05:15:52.824656 23215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:52.824926 23215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:52.826098 23215 hybrid_clock.cc:648] HybridClock initialized: now 1786511752826094 us; error 39 us; skew 500 ppm
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:52.828807 23215 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:52.829298 23215 webserver.cc:460] Webserver started at http://127.18.166.195:39799/ using document root <none> and password file <none>
I20260812 05:15:52.829586 23215 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:52.829639 23215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:52.829813 23215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:52.830960 23215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "0dde8e468fb642a49f61e6fb6ed7a65e"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "d788ca7011205bbea408e49c7f2fb018"
server_key_iv: "bbcfabefbda124ca810b62d2b4c868ef"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.831384 23215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "0dde8e468fb642a49f61e6fb6ed7a65e"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "d788ca7011205bbea408e49c7f2fb018"
server_key_iv: "bbcfabefbda124ca810b62d2b4c868ef"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.832890 23215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:15:52.833799 23230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.833999 23215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:52.834100 23215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "0dde8e468fb642a49f61e6fb6ed7a65e"
format_stamp: "Formatted at 2026-08-12 05:15:52 on dist-test-slave-srgv"
server_key: "d788ca7011205bbea408e49c7f2fb018"
server_key_iv: "bbcfabefbda124ca810b62d2b4c868ef"
server_key_version: "encryptionkey@0"
I20260812 05:15:52.834182 23215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:52.854315 23215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:52.856428 23215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:52.856559 23215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:52.856889 23215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:52.857240 23215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:52.857301 23215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.857348 23215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:52.857373 23215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:52.863956 23215 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:38123
I20260812 05:15:52.863998 23351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:38123 every 8 connection(s)
I20260812 05:15:52.864504 23215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:52.874492 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23215
I20260812 05:15:52.874614 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:52.874809 19099 external_mini_cluster.cc:1428] Setting key fda2e05a3b0a71948e22ceb655059a32
I20260812 05:15:52.874879 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.865393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:50415 (local address 127.18.166.254:43935)
0812 05:15:52.865458 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:52.865460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:52.865667 (+ 207us) server_negotiation.cc:407] Connection header received
0812 05:15:52.866679 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:52.866680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:52.866694 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:52.866758 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:52.867433 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.867924 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.869001 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.869224 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.869287 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.871839 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:52.871841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:52.871842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:52.871850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:52.873247 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.873806 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.873807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.873807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.873882 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.874140 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.874140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.874141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.874292 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:52.874371 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:52.874681 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 05:15:52.874785 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:15:52.875182 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.865354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43935 (local address 127.18.166.195:50415)
0812 05:15:52.865547 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:52.865590 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:52.866549 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:52.866826 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:52.866829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:52.867144 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:52.867350 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:52.867355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.868072 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.868073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:52.868908 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:52.868910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.869338 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.869339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:52.869570 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.870218 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:52.870227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:52.871713 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:52.873350 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:52.873359 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:52.873361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:52.873689 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:52.873948 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:52.873949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:52.873950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:52.874069 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:52.874365 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:52.874370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:52.874506 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 05:15:52.874620 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 05:15:52.874803 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":118,"threads_started":1}
I20260812 05:15:52.875735 23352 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43935
I20260812 05:15:52.875849 23352 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:52.876065 23352 heartbeater.cc:511] Master 127.18.166.254:43935 requested a full tablet report, sending...
I20260812 05:15:52.876413 22869 ts_manager.cc:194] Registered new tserver with Master: 0dde8e468fb642a49f61e6fb6ed7a65e (127.18.166.195:38123)
I20260812 05:15:52.876786 22869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:50415
I20260812 05:15:52.887086 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:52.912441 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.903771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.18.166.193:33585)
0812 05:15:52.903960 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:52.903964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:52.903976 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:15:52.904003 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:52.904005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:52.904021 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:52.904122 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:52.904775 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.905631 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.906408 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.906760 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.906873 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.909334 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:52.909344 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:52.909344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:52.909352 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:52.911325 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.911697 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.911700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.911700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.911759 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.911942 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.911943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.911943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.912094 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:52.912202 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:52.912272 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 05:15:52.912341 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
Aug 12 05:15:52 dist-test-slave-srgv krb5kdc[22823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:52.921265 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:52.914119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.18.166.254:43935)
0812 05:15:52.914188 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:52.914190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:52.914198 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:15:52.914233 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:52.914234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:52.914244 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:52.914294 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:52.914781 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.915269 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.915942 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:52.916122 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:52.916189 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:52.918600 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:52.918601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:52.918602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:52.918609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:52.920097 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.920465 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.920466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.920467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.920523 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:52.920737 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:52.920737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:52.920738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:52.920896 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:52.920972 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:52.921114 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 05:15:52.921179 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 05:15:52.921583 22869 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53924
I20260812 05:15:52.921993 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22936
I20260812 05:15:52.928017 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23076
I20260812 05:15:52.935839 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23215
I20260812 05:15:52.944195 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 22837
2026-08-12T05:15:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1971 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:53 dist-test-slave-srgv krb5kdc[23364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:53 dist-test-slave-srgv krb5kdc[23364](info): set up 2 sockets
Aug 12 05:15:53 dist-test-slave-srgv krb5kdc[23364](info): commencing operation
krb5kdc: starting...
W20260812 05:15:54.019690 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:54.148795 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43957
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:43957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:54.234933 23377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:54.235172 23377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:54.235198 23377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:54.236421 23377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:54.236469 23377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:54.236485 23377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:54.236497 23377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:54.236509 23377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:54.238225 23377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:43957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:54.238466 23377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:54.238668 23377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:54.243830 23385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.243839 23382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.243829 23383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:54.243854 23377 server_base.cc:1034] running on GCE node
I20260812 05:15:54.244362 23377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:54.244743 23377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:54.245896 23377 hybrid_clock.cc:648] HybridClock initialized: now 1786511754245882 us; error 33 us; skew 500 ppm
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:54.248601 23377 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:54.249027 23377 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:54.249082 23377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:54.249190 23377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:54.250383 23377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "61498546253b483899188b6f26838a6f"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "81e43b3a9bb23e2e0453a77cfad71402"
server_key_iv: "d13b5c737e2efb4f7bcb1fe3aa4f5fce"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.250710 23377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "61498546253b483899188b6f26838a6f"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "81e43b3a9bb23e2e0453a77cfad71402"
server_key_iv: "d13b5c737e2efb4f7bcb1fe3aa4f5fce"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.252107 23377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:54.252879 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.253084 23377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:54.253170 23377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "61498546253b483899188b6f26838a6f"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "81e43b3a9bb23e2e0453a77cfad71402"
server_key_iv: "d13b5c737e2efb4f7bcb1fe3aa4f5fce"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.253242 23377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:54.264699 23377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:54.266885 23377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:54.267019 23377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:54.271605 23377 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:43957
I20260812 05:15:54.271636 23447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:43957 every 8 connection(s)
I20260812 05:15:54.271974 23377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:54.273147 23448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:54.274567 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23377
I20260812 05:15:54.274698 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:54.274893 19099 external_mini_cluster.cc:1428] Setting key abce1110b19814042e798d56d0fd3e28
I20260812 05:15:54.276257 23448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f: Bootstrap starting.
I20260812 05:15:54.277055 23448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:54.277491 23448 log.cc:824] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:54.278363 23448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f: No bootstrap required, opened a new log
I20260812 05:15:54.280851 23448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498546253b483899188b6f26838a6f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43957 } }
I20260812 05:15:54.281073 23448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:54.281104 23448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61498546253b483899188b6f26838a6f, State: Initialized, Role: FOLLOWER
I20260812 05:15:54.281265 23448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498546253b483899188b6f26838a6f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43957 } }
I20260812 05:15:54.281348 23448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:54.281383 23448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:54.281459 23448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:54.282145 23448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498546253b483899188b6f26838a6f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43957 } }
I20260812 05:15:54.282246 23448 leader_election.cc:302] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61498546253b483899188b6f26838a6f; no voters:
I20260812 05:15:54.282452 23448 leader_election.cc:288] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:54.282495 23453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:54.282614 23453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [term 1 LEADER]: Becoming Leader. State: Replica: 61498546253b483899188b6f26838a6f, State: Running, Role: LEADER
I20260812 05:15:54.282704 23453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498546253b483899188b6f26838a6f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43957 } }
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:54.282805 23448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:54.283064 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61498546253b483899188b6f26838a6f. Latest consensus state: current_term: 1 leader_uuid: "61498546253b483899188b6f26838a6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498546253b483899188b6f26838a6f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43957 } } }
I20260812 05:15:54.283143 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:54.283079 23454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61498546253b483899188b6f26838a6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498546253b483899188b6f26838a6f" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43957 } } }
I20260812 05:15:54.283205 23454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:54.283355 23457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:54.283957 23457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:54.286620 23457 catalog_manager.cc:1347] Generated new cluster ID: 7755db78fa5e4be6af2c1b8e60f5920e
I20260812 05:15:54.286676 23457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:54.287606 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.275527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52276 (local address 127.18.166.254:43957)
0812 05:15:54.275753 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.275764 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.275806 (+ 42us) server_negotiation.cc:407] Connection header received
0812 05:15:54.276397 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.276408 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.276478 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.276661 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:54.277285 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.278724 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.279529 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.280253 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.280405 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.283450 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:54.283455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:54.283455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:54.283469 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:54.285694 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.286209 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.286213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.286213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.286331 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.286598 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.286599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.286600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.286932 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:54.287073 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.287236 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.287384 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260812 05:15:54.299410 23457 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:54.300010 23457 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:54.305032 23457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f: Generated new TSK 0
I20260812 05:15:54.305284 23457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:54.340982 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43957
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:54.430387 23474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:54.430644 23474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:54.430676 23474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:54.432014 23474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:54.432073 23474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:54.432111 23474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:54.433970 23474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:54.434227 23474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:54.434450 23474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:54.440049 23480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:54.440213 23474 server_base.cc:1034] running on GCE node
W20260812 05:15:54.440038 23482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.440038 23479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:54.440598 23474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:54.440891 23474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:54.442050 23474 hybrid_clock.cc:648] HybridClock initialized: now 1786511754442029 us; error 39 us; skew 500 ppm
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:54.445138 23474 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:54.445670 23474 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:54.445757 23474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:54.445931 23474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:54.447165 23474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "5beb1e27517a4606aecb73cbc78345f5"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "bcb5dc60265937562004151e3deb2d22"
server_key_iv: "107bdf0c80334e5edce941ba6b6c6a3d"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.447597 23474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "5beb1e27517a4606aecb73cbc78345f5"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "bcb5dc60265937562004151e3deb2d22"
server_key_iv: "107bdf0c80334e5edce941ba6b6c6a3d"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.449098 23474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:15:54.450109 23488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.450312 23474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:54.450431 23474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "5beb1e27517a4606aecb73cbc78345f5"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "bcb5dc60265937562004151e3deb2d22"
server_key_iv: "107bdf0c80334e5edce941ba6b6c6a3d"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.450520 23474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:54.460865 23474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:54.463369 23474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:54.463573 23474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:54.463852 23474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:54.464239 23474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:54.464303 23474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.464351 23474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:54.464377 23474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.471110 23474 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:44985
I20260812 05:15:54.471150 23607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:44985 every 8 connection(s)
I20260812 05:15:54.471525 23474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:54.476994 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23474
I20260812 05:15:54.477099 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:54.477361 19099 external_mini_cluster.cc:1428] Setting key 969ff64a0c731d7c0a2e3f3417c10708
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:54.482945 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.472427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:44479 (local address 127.18.166.254:43957)
0812 05:15:54.472502 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.472504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.472756 (+ 252us) server_negotiation.cc:407] Connection header received
0812 05:15:54.473290 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.473292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.473302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.473370 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:54.474517 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.475036 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.476208 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.476430 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.476490 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.479333 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:54.479336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:54.479336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:54.479346 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:54.480994 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.481671 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.481672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.481672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.481736 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.482093 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.482094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.482094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.482217 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:54.482296 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.482696 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.482852 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:15:54.483259 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.472396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43957 (local address 127.18.166.193:44479)
0812 05:15:54.472611 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:54.472660 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:54.473187 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:54.473504 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:54.473507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:54.473818 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:54.474398 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.474405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.475173 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.475174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.476102 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.476105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.476523 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.476524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.476643 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.477367 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:54.477380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:54.479218 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:54.481118 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:54.481131 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:54.481135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:54.481573 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:54.481815 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:54.481816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:54.481817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:54.482018 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:54.482294 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:54.482303 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:54.482509 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 05:15:54.482650 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 05:15:54.482972 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":126,"threads_started":1}
I20260812 05:15:54.483923 23608 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43957
I20260812 05:15:54.484045 23608 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:54.484272 23608 heartbeater.cc:511] Master 127.18.166.254:43957 requested a full tablet report, sending...
I20260812 05:15:54.484870 23410 ts_manager.cc:194] Registered new tserver with Master: 5beb1e27517a4606aecb73cbc78345f5 (127.18.166.193:44985)
I20260812 05:15:54.485591 23410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:44479
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:54.509207 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43957
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:54.709929 23613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:54.710737 23613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:54.710835 23613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:54.715322 23613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:54.715564 23613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:54.715680 23613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:54.721840 23613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:54.722863 23613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:54.723765 23613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:54.738741 23619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.738821 23621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.738821 23618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:54.740666 23613 server_base.cc:1034] running on GCE node
I20260812 05:15:54.741720 23613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:54.742724 23613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:54.744143 23613 hybrid_clock.cc:648] HybridClock initialized: now 1786511754744036 us; error 134 us; skew 500 ppm
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:54.754406 23613 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:54.755810 23613 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:54.755949 23613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:54.756309 23613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:54.758471 23613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "6836a05f7ff55551bcf843d8c273dbe4"
server_key_iv: "16e3d031dd33e5b79a71cb064ad621c9"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.758838 23613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "6836a05f7ff55551bcf843d8c273dbe4"
server_key_iv: "16e3d031dd33e5b79a71cb064ad621c9"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.760270 23613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:54.761116 23627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.761340 23613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:54.761497 23613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "6836a05f7ff55551bcf843d8c273dbe4"
server_key_iv: "16e3d031dd33e5b79a71cb064ad621c9"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.761595 23613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:54.775918 23613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:54.778681 23613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:54.778889 23613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:54.779168 23613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:54.779547 23613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:54.779613 23613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.779660 23613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:54.779707 23613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.787007 23613 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:34787
I20260812 05:15:54.787053 23744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:34787 every 8 connection(s)
I20260812 05:15:54.787381 23613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:54.790483 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23613
I20260812 05:15:54.790598 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:54.790803 19099 external_mini_cluster.cc:1428] Setting key 421c8a7555df7f7b96d269f2e859f1ce
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:54.798998 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.788272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:54715 (local address 127.18.166.254:43957)
0812 05:15:54.788350 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.788352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.788597 (+ 245us) server_negotiation.cc:407] Connection header received
0812 05:15:54.789094 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.789096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.789108 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.789166 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:54.790417 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.791211 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.792352 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.792576 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.792668 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.795589 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:54.795592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:54.795592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:54.795615 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:54.797221 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.797835 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.797837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.797838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.797911 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.798205 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.798206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.798207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.798341 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:54.798423 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.798791 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.798895 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:15:54.799223 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.788256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43957 (local address 127.18.166.194:54715)
0812 05:15:54.788453 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:54.788496 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:54.789002 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:54.789260 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:54.789264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:54.789668 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:54.790234 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.790241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.791323 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.791324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.792220 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.792223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.792680 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.792682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.792833 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.793526 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:54.793536 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:54.795433 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:54.797358 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:54.797373 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:54.797375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:54.797739 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:54.797988 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:54.797989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:54.797989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:54.798127 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:54.798412 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:54.798421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:54.798590 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 05:15:54.798753 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 05:15:54.798954 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:15:54.800149 23745 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43957
I20260812 05:15:54.800287 23745 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:54.800467 23745 heartbeater.cc:511] Master 127.18.166.254:43957 requested a full tablet report, sending...
I20260812 05:15:54.800889 23410 ts_manager.cc:194] Registered new tserver with Master: 0d310ee5b2774ac2b3a8b7b61b90f76a (127.18.166.194:34787)
I20260812 05:15:54.801366 23410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:54715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:54.819937 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43957
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:54.905894 23750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:54.906208 23750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:54.906260 23750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:54.907642 23750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:54.907749 23750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:54.907802 23750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:54.909610 23750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:54.909955 23750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:54.910238 23750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:54.915690 23756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.915674 23755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:54.915674 23758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:54.915848 23750 server_base.cc:1034] running on GCE node
I20260812 05:15:54.916181 23750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:54.916507 23750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:54.917682 23750 hybrid_clock.cc:648] HybridClock initialized: now 1786511754917664 us; error 59 us; skew 500 ppm
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:54.920506 23750 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:54.920915 23750 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:54.920997 23750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:54.921160 23750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:54.922394 23750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "1dc8aad762de4fa4a91c74d06a941695"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "8307de4623a5a98e7de620ea7ea87ae7"
server_key_iv: "ab37f5c07ccdebe8c5dbd5f692ec0eb1"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.922811 23750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "1dc8aad762de4fa4a91c74d06a941695"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "8307de4623a5a98e7de620ea7ea87ae7"
server_key_iv: "ab37f5c07ccdebe8c5dbd5f692ec0eb1"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.924335 23750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:15:54.925199 23764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.925465 23750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:54.925575 23750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "1dc8aad762de4fa4a91c74d06a941695"
format_stamp: "Formatted at 2026-08-12 05:15:54 on dist-test-slave-srgv"
server_key: "8307de4623a5a98e7de620ea7ea87ae7"
server_key_iv: "ab37f5c07ccdebe8c5dbd5f692ec0eb1"
server_key_version: "encryptionkey@0"
I20260812 05:15:54.925640 23750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:54.934196 23750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:54.936861 23750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:54.936996 23750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:54.937213 23750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:54.937618 23750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:54.937662 23750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.937686 23750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:54.937721 23750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:54.944288 23750 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:40925
I20260812 05:15:54.944372 23882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:40925 every 8 connection(s)
I20260812 05:15:54.944715 23750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:54.945746 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23750
I20260812 05:15:54.945868 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:54.946094 19099 external_mini_cluster.cc:1428] Setting key a92df46c098f83a457cc0ac0548250cd
Aug 12 05:15:54 dist-test-slave-srgv krb5kdc[23364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:54.955657 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.945994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:44309 (local address 127.18.166.254:43957)
0812 05:15:54.946075 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.946078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.946147 (+ 69us) server_negotiation.cc:407] Connection header received
0812 05:15:54.946730 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.946732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.946746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.946796 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:54.947846 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.948359 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.949487 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.949686 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.949772 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.952345 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:54.952347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:54.952348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:54.952359 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:54.954032 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.954522 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.954524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.954524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.954603 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.954942 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.954943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.954944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.955100 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:54.955181 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.955481 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.955586 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:15:54.955801 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.945658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43957 (local address 127.18.166.195:44309)
0812 05:15:54.945977 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:54.946053 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:54.946605 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:54.946880 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:54.946885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:54.947195 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:54.947729 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.947736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.948475 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.948476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.949328 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.949330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.949786 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.949787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.949934 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.950626 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:54.950635 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:54.952212 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:54.954138 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:54.954150 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:54.954154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:54.954430 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:54.954677 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:54.954678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:54.954678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:54.954819 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:54.955180 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:54.955185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:54.955326 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 05:15:54.955420 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 05:15:54.955567 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260812 05:15:54.956418 23883 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43957
I20260812 05:15:54.956573 23883 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:54.956771 23883 heartbeater.cc:511] Master 127.18.166.254:43957 requested a full tablet report, sending...
I20260812 05:15:54.957273 23410 ts_manager.cc:194] Registered new tserver with Master: 1dc8aad762de4fa4a91c74d06a941695 (127.18.166.195:40925)
I20260812 05:15:54.957788 23410 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:44309
I20260812 05:15:54.958371 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:15:54.966758 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.959871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.18.166.254:43957)
0812 05:15:54.959955 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.959957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.960018 (+ 61us) server_negotiation.cc:407] Connection header received
0812 05:15:54.960071 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.960071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.960082 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.960137 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:54.960738 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.961230 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.962035 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.962223 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.962282 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.963606 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:54.963608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:54.963608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:54.963615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:54.965350 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.965811 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.965813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.965813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.965883 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:54.966104 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:54.966105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:54.966105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:54.966291 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:54.966411 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.966550 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.966648 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 05:15:54.968464 23410 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:15:54.970328 23410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:54.977643 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.973156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:40925 (local address 127.0.0.1:59890)
0812 05:15:54.973496 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:54.973506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:54.973560 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:54.973800 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:54.973803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:54.973805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:54.973900 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.973903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.975330 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.975332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.976383 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.976387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.977259 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.977260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.977358 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.977374 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:15:54.977477 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 05:15:54.977540 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":87,"thread_start_us":111,"threads_started":1}
I20260812 05:15:54.977692 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.973294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.18.166.195:40925)
0812 05:15:54.973573 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.973579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.973621 (+ 42us) server_negotiation.cc:407] Connection header received
0812 05:15:54.973668 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.973670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.973688 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.973782 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:54.973989 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.975204 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.976499 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.977041 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.977150 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.977219 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.977509 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.977573 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":152,"threads_started":1}
I20260812 05:15:54.978174 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.973093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:34787 (local address 127.0.0.1:35684)
0812 05:15:54.973374 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:54.973471 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:54.973554 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:54.974175 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:54.974176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:54.974178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:54.974247 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.974249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.975486 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.975487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.976739 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.976743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.977673 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.977674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.977749 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.977754 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:15:54.978006 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 05:15:54.978089 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":66,"threads_started":1}
I20260812 05:15:54.978618 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.973310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44985 (local address 127.0.0.1:54310)
0812 05:15:54.973768 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:54.973779 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:54.973838 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:54.974327 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:54.974328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:54.974330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:54.974374 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.974375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.975302 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.975303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.977020 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:54.977023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.978333 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.978335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:54.978414 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.978418 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:54.978474 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:15:54.978517 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":96,"thread_start_us":85,"threads_started":1}
I20260812 05:15:54.978762 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.973256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.18.166.194:34787)
0812 05:15:54.973989 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.973993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.974003 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:54.974043 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.974045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.974061 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.974173 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:54.974314 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.975376 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.976852 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.977566 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.978484 (+ 918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.978566 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.978611 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.978666 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":109,"threads_started":1}
I20260812 05:15:54.978891 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:54.973577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.18.166.193:44985)
0812 05:15:54.973721 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:54.973725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:54.974034 (+ 309us) server_negotiation.cc:407] Connection header received
0812 05:15:54.974085 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:54.974088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:54.974106 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:54.974184 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:54.974452 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.975176 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.977127 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:54.978011 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:54.978098 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:54.978154 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:54.978736 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 05:15:54.978795 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260812 05:15:54.978762 23814 tablet_service.cc:1467] Processing CreateTablet for tablet a422f2fd50104b6eac0b0b8b020dd9f6 (DEFAULT_TABLE table=test-table [id=a495b92a73854403bd27471282ff0117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:54.979231 23814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a422f2fd50104b6eac0b0b8b020dd9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:54.979475 23676 tablet_service.cc:1467] Processing CreateTablet for tablet a422f2fd50104b6eac0b0b8b020dd9f6 (DEFAULT_TABLE table=test-table [id=a495b92a73854403bd27471282ff0117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:54.979580 23537 tablet_service.cc:1467] Processing CreateTablet for tablet a422f2fd50104b6eac0b0b8b020dd9f6 (DEFAULT_TABLE table=test-table [id=a495b92a73854403bd27471282ff0117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:54.979842 23676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a422f2fd50104b6eac0b0b8b020dd9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:54.979938 23537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a422f2fd50104b6eac0b0b8b020dd9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:54.982981 23900 tablet_bootstrap.cc:492] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: Bootstrap starting.
I20260812 05:15:54.983563 23901 tablet_bootstrap.cc:492] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Bootstrap starting.
I20260812 05:15:54.983974 23900 tablet_bootstrap.cc:654] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:54.983995 23902 tablet_bootstrap.cc:492] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: Bootstrap starting.
I20260812 05:15:54.984383 23901 tablet_bootstrap.cc:654] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:54.984798 23900 log.cc:824] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:54.984978 23902 tablet_bootstrap.cc:654] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:54.984979 23901 log.cc:824] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:54.985347 23902 log.cc:824] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:54.985965 23901 tablet_bootstrap.cc:492] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: No bootstrap required, opened a new log
I20260812 05:15:54.986050 23901 ts_tablet_manager.cc:1397] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:54.986107 23900 tablet_bootstrap.cc:492] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: No bootstrap required, opened a new log
I20260812 05:15:54.986161 23902 tablet_bootstrap.cc:492] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: No bootstrap required, opened a new log
I20260812 05:15:54.986183 23900 ts_tablet_manager.cc:1397] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:15:54.986224 23902 ts_tablet_manager.cc:1397] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:15:54.987871 23900 raft_consensus.cc:348] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:54.988087 23900 raft_consensus.cc:374] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:54.988138 23900 raft_consensus.cc:729] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d310ee5b2774ac2b3a8b7b61b90f76a, State: Initialized, Role: FOLLOWER
I20260812 05:15:54.988260 23900 consensus_queue.cc:260] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:54.988310 23901 raft_consensus.cc:348] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:54.988497 23901 raft_consensus.cc:374] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:54.988549 23901 raft_consensus.cc:729] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc8aad762de4fa4a91c74d06a941695, State: Initialized, Role: FOLLOWER
I20260812 05:15:54.988641 23900 ts_tablet_manager.cc:1428] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:15:54.988759 23745 heartbeater.cc:503] Master 127.18.166.254:43957 was elected leader, sending a full tablet report...
I20260812 05:15:54.988713 23901 consensus_queue.cc:260] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:54.988802 23902 raft_consensus.cc:348] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:54.988965 23902 raft_consensus.cc:374] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:54.989012 23901 ts_tablet_manager.cc:1428] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:15:54.989017 23902 raft_consensus.cc:729] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5beb1e27517a4606aecb73cbc78345f5, State: Initialized, Role: FOLLOWER
I20260812 05:15:54.989159 23883 heartbeater.cc:503] Master 127.18.166.254:43957 was elected leader, sending a full tablet report...
I20260812 05:15:54.989372 23902 consensus_queue.cc:260] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:54.989725 23608 heartbeater.cc:503] Master 127.18.166.254:43957 was elected leader, sending a full tablet report...
I20260812 05:15:54.989714 23902 ts_tablet_manager.cc:1428] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260812 05:15:55.038031 23746 tablet.cc:2369] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:15:55.090293 23907 raft_consensus.cc:482] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:55.090516 23907 raft_consensus.cc:504] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:55.090910 23907 leader_election.cc:288] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d310ee5b2774ac2b3a8b7b61b90f76a (127.18.166.194:34787), 5beb1e27517a4606aecb73cbc78345f5 (127.18.166.193:44985)
I20260812 05:15:55.094216 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.091421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:35949 (local address 127.18.166.193:44985)
0812 05:15:55.091484 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.091486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.091600 (+ 114us) server_negotiation.cc:407] Connection header received
0812 05:15:55.091664 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.091666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.091676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.091743 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:55.091881 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.092436 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.093497 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.093843 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.093899 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:55.093906 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.094090 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.094136 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:15:55.094270 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.091285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39969 (local address 127.18.166.194:34787)
0812 05:15:55.091352 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.091355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.091367 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:15:55.091400 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.091401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.091410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.091494 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:15:55.091706 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.092333 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.093453 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.093864 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.093963 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:55.093971 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.094151 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.094195 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:15:55.094310 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.091170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:34787 (local address 127.18.166.195:39969)
0812 05:15:55.091234 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.091248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.091298 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.091507 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.091508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.091510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:55.091620 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:55.091622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.092424 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.092425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:55.093333 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:55.093336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.093978 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.093979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:55.094065 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:55.094069 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.094126 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.094161 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 05:15:55.094233 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.091313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44985 (local address 127.18.166.195:35949)
0812 05:15:55.091516 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.091529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.091599 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.091744 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.091744 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.091746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:15:55.091812 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:55.091814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.092538 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.092539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:55.093363 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:55.093365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.093927 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.093928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:55.094003 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:55.094007 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.094061 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.094098 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":132,"threads_started":1}
I20260812 05:15:55.094821 23558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a422f2fd50104b6eac0b0b8b020dd9f6" candidate_uuid: "1dc8aad762de4fa4a91c74d06a941695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5beb1e27517a4606aecb73cbc78345f5" is_pre_election: true
I20260812 05:15:55.094834 23697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a422f2fd50104b6eac0b0b8b020dd9f6" candidate_uuid: "1dc8aad762de4fa4a91c74d06a941695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" is_pre_election: true
I20260812 05:15:55.095016 23558 raft_consensus.cc:2454] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc8aad762de4fa4a91c74d06a941695 in term 0.
I20260812 05:15:55.095036 23697 raft_consensus.cc:2454] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc8aad762de4fa4a91c74d06a941695 in term 0.
I20260812 05:15:55.095293 23767 leader_election.cc:302] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d310ee5b2774ac2b3a8b7b61b90f76a, 1dc8aad762de4fa4a91c74d06a941695; no voters:
I20260812 05:15:55.095427 23907 raft_consensus.cc:2781] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:55.095485 23907 raft_consensus.cc:482] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:55.095510 23907 raft_consensus.cc:3037] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.096230 23907 raft_consensus.cc:504] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:55.096370 23907 leader_election.cc:288] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [CANDIDATE]: Term 1 election: Requested vote from peers 0d310ee5b2774ac2b3a8b7b61b90f76a (127.18.166.194:34787), 5beb1e27517a4606aecb73cbc78345f5 (127.18.166.193:44985)
I20260812 05:15:55.096513 23697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a422f2fd50104b6eac0b0b8b020dd9f6" candidate_uuid: "1dc8aad762de4fa4a91c74d06a941695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a"
I20260812 05:15:55.096561 23558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a422f2fd50104b6eac0b0b8b020dd9f6" candidate_uuid: "1dc8aad762de4fa4a91c74d06a941695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5beb1e27517a4606aecb73cbc78345f5"
I20260812 05:15:55.096616 23697 raft_consensus.cc:3037] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.096674 23558 raft_consensus.cc:3037] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.097362 23697 raft_consensus.cc:2454] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dc8aad762de4fa4a91c74d06a941695 in term 1.
I20260812 05:15:55.097383 23558 raft_consensus.cc:2454] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dc8aad762de4fa4a91c74d06a941695 in term 1.
I20260812 05:15:55.097607 23767 leader_election.cc:302] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dc8aad762de4fa4a91c74d06a941695, 5beb1e27517a4606aecb73cbc78345f5; no voters:
I20260812 05:15:55.097728 23907 raft_consensus.cc:2781] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:55.097921 23907 raft_consensus.cc:686] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 1 LEADER]: Becoming Leader. State: Replica: 1dc8aad762de4fa4a91c74d06a941695, State: Running, Role: LEADER
I20260812 05:15:55.098071 23907 consensus_queue.cc:237] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } }
I20260812 05:15:55.099035 23409 catalog_manager.cc:5495] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dc8aad762de4fa4a91c74d06a941695 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "1dc8aad762de4fa4a91c74d06a941695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dc8aad762de4fa4a91c74d06a941695" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 40925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 } health_report { overall_health: UNKNOWN } } }
I20260812 05:15:55.126791 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.124243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.18.166.195:40925)
0812 05:15:55.124332 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.124334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.124344 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:55.124382 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.124383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.124402 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.124484 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:15:55.124726 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.125210 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.125977 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:55.126201 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:55.126262 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:55.126462 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:15:55.126576 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:15:55.126642 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.126684 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.126703 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 05:15:55.129346 23697 raft_consensus.cc:1261] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 1 FOLLOWER]: Refusing update from remote peer 1dc8aad762de4fa4a91c74d06a941695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:55.129351 23558 raft_consensus.cc:1261] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5 [term 1 FOLLOWER]: Refusing update from remote peer 1dc8aad762de4fa4a91c74d06a941695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:55.129771 23910 consensus_queue.cc:1035] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5beb1e27517a4606aecb73cbc78345f5" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:55.130012 23907 consensus_queue.cc:1035] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d310ee5b2774ac2b3a8b7b61b90f76a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:55.134162 23914 mvcc.cc:205] Tried to move back new op lower bound from 7317552149006467072 to 7317552148882554880. Current Snapshot: MvccSnapshot[applied={T|T < 7317552149006467072}]
I20260812 05:15:55.135238 23915 mvcc.cc:205] Tried to move back new op lower bound from 7317552149006467072 to 7317552148882554880. Current Snapshot: MvccSnapshot[applied={T|T < 7317552149006467072 or (T in {7317552149006467072})}]
I20260812 05:15:55.135500 23409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:55.135617 23409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:55.136579 23409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61498546253b483899188b6f26838a6f: Sending DeleteTablet for 3 replicas of tablet a422f2fd50104b6eac0b0b8b020dd9f6
I20260812 05:15:55.137017 23676 tablet_service.cc:1514] Processing DeleteTablet for tablet a422f2fd50104b6eac0b0b8b020dd9f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:55 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:35684
I20260812 05:15:55.137094 23537 tablet_service.cc:1514] Processing DeleteTablet for tablet a422f2fd50104b6eac0b0b8b020dd9f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:55 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:54310
I20260812 05:15:55.137074 23814 tablet_service.cc:1514] Processing DeleteTablet for tablet a422f2fd50104b6eac0b0b8b020dd9f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:55 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59890
I20260812 05:15:55.137372 23923 tablet_replica.cc:330] T a422f2fd50104b6eac0b0b8b020dd9f6 P 5beb1e27517a4606aecb73cbc78345f5: stopping tablet replica
I20260812 05:15:55.137624 23922 tablet_replica.cc:330] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a: stopping tablet replica
I20260812 05:15:55.137781 23922 raft_consensus.cc:2229] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:55.137840 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23474
I20260812 05:15:55.140676 23924 tablet_replica.cc:330] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: stopping tablet replica
I20260812 05:15:55.140861 23924 raft_consensus.cc:2229] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:55.140988 23924 raft_consensus.cc:2258] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:55.146018 23922 raft_consensus.cc:2258] T a422f2fd50104b6eac0b0b8b020dd9f6 P 0d310ee5b2774ac2b3a8b7b61b90f76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:55.146719 23924 ts_tablet_manager.cc:1905] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 05:15:55.147120 23395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:44985 (error 108)
I20260812 05:15:55.147804 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.147631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44985 (local address 127.0.0.1:54324)
0812 05:15:55.147702 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.147758 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:44985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 05:15:55.147945 23395 catalog_manager.cc:4584] TS 5beb1e27517a4606aecb73cbc78345f5 (127.18.166.193:44985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a422f2fd50104b6eac0b0b8b020dd9f6: Network error: Client connection negotiation failed: client connection to 127.18.166.193:44985: connect: Connection refused (error 111)
I20260812 05:15:55.148298 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23613
I20260812 05:15:55.151106 23924 ts_tablet_manager.cc:1918] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:15:55.151229 23924 log.cc:1196] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/a422f2fd50104b6eac0b0b8b020dd9f6
I20260812 05:15:55.151513 23924 ts_tablet_manager.cc:1939] T a422f2fd50104b6eac0b0b8b020dd9f6 P 1dc8aad762de4fa4a91c74d06a941695: Deleting consensus metadata
I20260812 05:15:55.152419 23392 catalog_manager.cc:4841] TS 1dc8aad762de4fa4a91c74d06a941695 (127.18.166.195:40925): tablet a422f2fd50104b6eac0b0b8b020dd9f6 (table test-table [id=a495b92a73854403bd27471282ff0117]) successfully deleted
I20260812 05:15:55.159247 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23750
I20260812 05:15:55.159468 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.159328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:34787 (local address 127.0.0.1:35698)
0812 05:15:55.159392 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.159421 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.194:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:15:55.166368 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23377
2026-08-12T05:15:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2223 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T05:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:55Z Disabled control of system clock
I20260812 05:15:55.199191 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:38199
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:38199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:15:55.286945 23929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:55.287196 23929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:55.287230 23929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:55.288440 23929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:55.288488 23929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:55.288501 23929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:55.288514 23929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:55.288527 23929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:55.289978 23929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:38199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:38199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:55.290215 23929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:55.290588 23929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:55.295920 23935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.296078 23929 server_base.cc:1034] running on GCE node
W20260812 05:15:55.295913 23938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.295935 23934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.296514 23929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:55.296868 23929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:55.298033 23929 hybrid_clock.cc:648] HybridClock initialized: now 1786511755298008 us; error 54 us; skew 500 ppm
I20260812 05:15:55.299590 23929 webserver.cc:460] Webserver started at http://127.18.166.254:39377/ using document root <none> and password file <none>
I20260812 05:15:55.299873 23929 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:55.299952 23929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:55.300115 23929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:55.301308 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "e8a4db377d2347428f1511a3e0fd7129"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "cccbda36c073002f51ad93313ab71d49"
server_key_iv: "8a5ecbe1ff5c2af1b21876f15de0fdd9"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.301783 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "e8a4db377d2347428f1511a3e0fd7129"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "cccbda36c073002f51ad93313ab71d49"
server_key_iv: "8a5ecbe1ff5c2af1b21876f15de0fdd9"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.303314 23929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:55.304185 23944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.304387 23929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:55.304507 23929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "e8a4db377d2347428f1511a3e0fd7129"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "cccbda36c073002f51ad93313ab71d49"
server_key_iv: "8a5ecbe1ff5c2af1b21876f15de0fdd9"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.304600 23929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:55.308777 23929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:55.309115 23929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:55.309288 23929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:55.313848 23929 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:38199
I20260812 05:15:55.313905 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:38199 every 8 connection(s)
I20260812 05:15:55.314328 23929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:55.314841 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 23929
I20260812 05:15:55.314954 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:55.315148 19099 external_mini_cluster.cc:1428] Setting key e6e1f01cea592a057b87b91b109d3763
I20260812 05:15:55.315246 24000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:55.318500 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.316200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.18.166.254:38199)
0812 05:15:55.316435 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.316442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.316472 (+ 30us) server_negotiation.cc:407] Connection header received
0812 05:15:55.316987 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.316994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.317039 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.317169 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.317981 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.317985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.317985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.317989 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.318033 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.318043 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.318101 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.318211 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.318294 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260812 05:15:55.319061 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129: Bootstrap starting.
I20260812 05:15:55.319737 24000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:55.320036 24000 log.cc:824] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:55.320879 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129: No bootstrap required, opened a new log
I20260812 05:15:55.322608 24000 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a4db377d2347428f1511a3e0fd7129" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 38199 } }
I20260812 05:15:55.322826 24000 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:55.322881 24000 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a4db377d2347428f1511a3e0fd7129, State: Initialized, Role: FOLLOWER
I20260812 05:15:55.323066 24000 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a4db377d2347428f1511a3e0fd7129" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 38199 } }
I20260812 05:15:55.323197 24000 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:55.323254 24000 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:55.323318 24000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.324132 24000 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a4db377d2347428f1511a3e0fd7129" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 38199 } }
I20260812 05:15:55.324286 24000 leader_election.cc:302] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8a4db377d2347428f1511a3e0fd7129; no voters:
I20260812 05:15:55.324559 24000 leader_election.cc:288] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:55.324967 24006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:55.324980 24000 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:55.325151 24006 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [term 1 LEADER]: Becoming Leader. State: Replica: e8a4db377d2347428f1511a3e0fd7129, State: Running, Role: LEADER
I20260812 05:15:55.325266 24006 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a4db377d2347428f1511a3e0fd7129" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 38199 } }
I20260812 05:15:55.325686 24008 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8a4db377d2347428f1511a3e0fd7129. Latest consensus state: current_term: 1 leader_uuid: "e8a4db377d2347428f1511a3e0fd7129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a4db377d2347428f1511a3e0fd7129" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 38199 } } }
I20260812 05:15:55.325706 24007 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8a4db377d2347428f1511a3e0fd7129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8a4db377d2347428f1511a3e0fd7129" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 38199 } } }
I20260812 05:15:55.325793 24008 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:55.325815 24007 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:55.326341 24015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:55.326871 24015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:55.328362 24015 catalog_manager.cc:1347] Generated new cluster ID: 8697287413054d768c1725d8833a2af3
I20260812 05:15:55.328409 24015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:55.356333 24015 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:55.356549 24015 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:55.368613 24015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129: Generated new TSK 0
I20260812 05:15:55.368858 24015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:15:55.370844 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:38199
--builtin_ntp_servers=127.18.166.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:15:55.462255 24026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:55.462495 24026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:55.462522 24026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:55.463837 24026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:55.463892 24026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:55.463922 24026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:55.465477 24026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:38199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:55.465798 24026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:55.466145 24026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:55.471433 24031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.471450 24035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.471591 24026 server_base.cc:1034] running on GCE node
W20260812 05:15:55.471467 24033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.471853 24026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:55.472090 24026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:55.473256 24026 hybrid_clock.cc:648] HybridClock initialized: now 1786511755473245 us; error 32 us; skew 500 ppm
I20260812 05:15:55.474670 24026 webserver.cc:460] Webserver started at http://127.18.166.193:43227/ using document root <none> and password file <none>
I20260812 05:15:55.474946 24026 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:55.474998 24026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:55.475186 24026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:55.476398 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "08396e417cd049b180da69bdb458a30c"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "72008c1092cfc361426f47aade1cfcf2"
server_key_iv: "f0d160dcf6feb5cbe018e90b2f55191f"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.476868 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "08396e417cd049b180da69bdb458a30c"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "72008c1092cfc361426f47aade1cfcf2"
server_key_iv: "f0d160dcf6feb5cbe018e90b2f55191f"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.478456 24026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:15:55.479276 24041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.479486 24026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:55.479583 24026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "08396e417cd049b180da69bdb458a30c"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "72008c1092cfc361426f47aade1cfcf2"
server_key_iv: "f0d160dcf6feb5cbe018e90b2f55191f"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.479648 24026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:55.483281 24026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:55.483542 24026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:55.483647 24026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:55.483820 24026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:55.484167 24026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:55.484205 24026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.484227 24026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:55.484241 24026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.490442 24026 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:42295
I20260812 05:15:55.490533 24159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:42295 every 8 connection(s)
I20260812 05:15:55.490823 24026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:55.494225 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.491610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:53929 (local address 127.18.166.254:38199)
0812 05:15:55.491699 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.491702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.491994 (+ 292us) server_negotiation.cc:407] Connection header received
0812 05:15:55.492545 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.492546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.492554 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.492607 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.493935 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.493937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.493937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.493944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.493961 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.493966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.494027 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.494127 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.494144 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 05:15:55.494477 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.491582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:38199 (local address 127.18.166.193:53929)
0812 05:15:55.491826 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.491893 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.492419 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.492687 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.492691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.492699 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.493769 (+ 1070us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.493769 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.493802 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.493836 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.494032 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.494039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.494043 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.494140 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.494218 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260812 05:15:55.495107 24160 heartbeater.cc:347] Connected to a master server at 127.18.166.254:38199
I20260812 05:15:55.495220 24160 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:55.495386 24160 heartbeater.cc:511] Master 127.18.166.254:38199 requested a full tablet report, sending...
I20260812 05:15:55.495951 23962 ts_manager.cc:194] Registered new tserver with Master: 08396e417cd049b180da69bdb458a30c (127.18.166.193:42295)
I20260812 05:15:55.496938 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24026
I20260812 05:15:55.497048 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:55.497248 19099 external_mini_cluster.cc:1428] Setting key 582aa63ab8e5e94b68456d80f436d6d8
I20260812 05:15:55.498286 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:38199
--builtin_ntp_servers=127.18.166.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:15:55.584949 24163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:55.585191 24163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:55.585216 24163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:55.586943 24163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:55.587049 24163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:55.587097 24163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:55.589608 24163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:38199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:55.589859 24163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:55.590142 24163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:55.595572 24171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.595582 24169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.595589 24173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.595743 24163 server_base.cc:1034] running on GCE node
I20260812 05:15:55.596130 24163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:55.596460 24163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:55.597651 24163 hybrid_clock.cc:648] HybridClock initialized: now 1786511755597632 us; error 30 us; skew 500 ppm
I20260812 05:15:55.599205 24163 webserver.cc:460] Webserver started at http://127.18.166.194:39297/ using document root <none> and password file <none>
I20260812 05:15:55.599499 24163 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:55.599581 24163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:55.599756 24163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:55.600981 24163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "d88cedf7321745f2bfa33dbb69a78fe3"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "af1e88cc6f779edca1b65186e34e7185"
server_key_iv: "0daed0203fe4bd0039b83704f17fb21c"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.601397 24163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "d88cedf7321745f2bfa33dbb69a78fe3"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "af1e88cc6f779edca1b65186e34e7185"
server_key_iv: "0daed0203fe4bd0039b83704f17fb21c"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.602952 24163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:55.603907 24179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.604127 24163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:15:55.604249 24163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "d88cedf7321745f2bfa33dbb69a78fe3"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "af1e88cc6f779edca1b65186e34e7185"
server_key_iv: "0daed0203fe4bd0039b83704f17fb21c"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.604337 24163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:55.608206 24163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:55.608549 24163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:55.608711 24163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:55.608958 24163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:55.609330 24163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:55.609366 24163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.609390 24163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:55.609465 24163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.615993 24163 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:35965
I20260812 05:15:55.616050 24296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:35965 every 8 connection(s)
I20260812 05:15:55.616449 24163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:55.619725 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.617360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:36611 (local address 127.18.166.254:38199)
0812 05:15:55.617464 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.617467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.617628 (+ 161us) server_negotiation.cc:407] Connection header received
0812 05:15:55.618107 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.618109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.618114 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.618172 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.619426 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.619428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.619428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.619433 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.619450 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.619455 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.619514 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.619638 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.619654 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 05:15:55.619971 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.617274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:38199 (local address 127.18.166.194:36611)
0812 05:15:55.617485 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.617535 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.618012 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.618229 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.618232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.618240 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.619251 (+ 1011us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.619252 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.619283 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.619316 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.619545 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.619552 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.619555 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.619639 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.619728 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260812 05:15:55.620522 24297 heartbeater.cc:347] Connected to a master server at 127.18.166.254:38199
I20260812 05:15:55.620613 24297 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:55.620792 24297 heartbeater.cc:511] Master 127.18.166.254:38199 requested a full tablet report, sending...
I20260812 05:15:55.621217 23962 ts_manager.cc:194] Registered new tserver with Master: d88cedf7321745f2bfa33dbb69a78fe3 (127.18.166.194:35965)
I20260812 05:15:55.624199 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24163
I20260812 05:15:55.624351 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:55.624537 19099 external_mini_cluster.cc:1428] Setting key 8534a2e6455db4f68b9c7bacc9645baf
I20260812 05:15:55.625746 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:38199
--builtin_ntp_servers=127.18.166.212:40573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:15:55.711591 24300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:55.711831 24300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:55.711858 24300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:55.713135 24300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:55.713194 24300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:55.713255 24300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:55.714798 24300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:38199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:55.715093 24300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:55.715441 24300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:55.720959 24305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.720976 24306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.720959 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.721035 24300 server_base.cc:1034] running on GCE node
I20260812 05:15:55.721526 24300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:55.721829 24300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:55.723011 24300 hybrid_clock.cc:648] HybridClock initialized: now 1786511755722998 us; error 51 us; skew 500 ppm
I20260812 05:15:55.724498 24300 webserver.cc:460] Webserver started at http://127.18.166.195:36381/ using document root <none> and password file <none>
I20260812 05:15:55.724805 24300 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:55.724884 24300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:55.725136 24300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:55.726543 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "31dfac2d8bf1449292984b3d4228f9c0"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "28e7a0ae75eb2a13bbf23f86b5ea7983"
server_key_iv: "791e1d4e328a7d9fafbced43dbe98473"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.726948 24300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "31dfac2d8bf1449292984b3d4228f9c0"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "28e7a0ae75eb2a13bbf23f86b5ea7983"
server_key_iv: "791e1d4e328a7d9fafbced43dbe98473"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.728410 24300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:15:55.729241 24316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.729497 24300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:55.729605 24300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "31dfac2d8bf1449292984b3d4228f9c0"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "28e7a0ae75eb2a13bbf23f86b5ea7983"
server_key_iv: "791e1d4e328a7d9fafbced43dbe98473"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.729707 24300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:55.733444 24300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:55.733786 24300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:55.733953 24300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:55.734205 24300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:55.734570 24300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:55.734629 24300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.734674 24300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:55.734702 24300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.741034 24300 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43607
I20260812 05:15:55.741117 24434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43607 every 8 connection(s)
I20260812 05:15:55.741629 24300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:15:55.744719 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.742487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:38791 (local address 127.18.166.254:38199)
0812 05:15:55.742560 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.742562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.742774 (+ 212us) server_negotiation.cc:407] Connection header received
0812 05:15:55.743266 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.743267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.743275 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.743330 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.744502 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.744503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.744503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.744508 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.744523 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.744527 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.744583 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.744643 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.744656 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:15:55.744930 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.742470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:38199 (local address 127.18.166.195:38791)
0812 05:15:55.742658 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.742694 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.743166 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.743385 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.743389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.743397 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.744336 (+ 939us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.744337 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.744367 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.744398 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.744593 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.744596 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.744598 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.744653 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.744705 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260812 05:15:55.745437 24435 heartbeater.cc:347] Connected to a master server at 127.18.166.254:38199
I20260812 05:15:55.745529 24435 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:55.745659 24435 heartbeater.cc:511] Master 127.18.166.254:38199 requested a full tablet report, sending...
I20260812 05:15:55.746050 23962 ts_manager.cc:194] Registered new tserver with Master: 31dfac2d8bf1449292984b3d4228f9c0 (127.18.166.195:43607)
I20260812 05:15:55.751438 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24300
I20260812 05:15:55.751579 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:55.751768 19099 external_mini_cluster.cc:1428] Setting key 02cd8a845fc1003991d815ac9fc053a9
I20260812 05:15:55.752776 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:15:55.755606 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.754329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.18.166.254:38199)
0812 05:15:55.754409 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.754411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.754516 (+ 105us) server_negotiation.cc:407] Connection header received
0812 05:15:55.754626 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.754627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.754634 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.754699 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.755338 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.755340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.755340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.755345 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.755363 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.755367 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.755440 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.755503 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.755520 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:15:55.757402 23962 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:15:55.759181 23962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:15:55.763490 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.761551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:35965 (local address 127.0.0.1:50654)
0812 05:15:55.761964 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.761976 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.762039 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.762515 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.762516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.762519 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.762973 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.762974 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.762982 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.762997 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.763306 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.763314 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.763318 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.763387 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.763402 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260812 05:15:55.763568 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.761743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.18.166.193:42295)
0812 05:15:55.762055 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.762059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.762069 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:55.762100 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.762102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.762114 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.762196 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.763261 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.763265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.763265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.763269 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.763299 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.763316 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.763381 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.763452 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.763467 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260812 05:15:55.763602 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.761563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43607 (local address 127.0.0.1:56624)
0812 05:15:55.762032 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.762042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.762076 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.762619 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.762620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.762622 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.763155 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.763155 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.763160 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.763167 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.763433 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.763434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.763435 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.763480 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.763490 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":111,"threads_started":1}
I20260812 05:15:55.763746 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.761767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.18.166.194:35965)
0812 05:15:55.761958 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.761963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.762102 (+ 139us) server_negotiation.cc:407] Connection header received
0812 05:15:55.762134 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.762136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.762148 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.762682 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.763105 (+ 423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.763114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.763114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.763122 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.763166 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.763197 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.763275 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.763631 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.763651 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260812 05:15:55.763850 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.761551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:42295 (local address 127.0.0.1:45412)
0812 05:15:55.761798 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.761848 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.761937 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.762187 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.762193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.762199 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.762805 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.762805 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.762823 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.762853 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.763389 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.763390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.763391 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.763448 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.763463 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":113,"threads_started":1}
I20260812 05:15:55.764413 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.761965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56624 (local address 127.18.166.195:43607)
0812 05:15:55.762177 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.762182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.762191 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:15:55.762215 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.762217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.762225 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.762309 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.763245 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.763254 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.763268 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.763273 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.763315 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.763334 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.763409 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.763524 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.763541 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260812 05:15:55.764706 24093 tablet_service.cc:1467] Processing CreateTablet for tablet 43a21f53ff1c48a8a7a8baca8011695a (DEFAULT_TABLE table=test-table [id=0a77262a3e284fe7a11b2c5edbb5ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:55.764765 24229 tablet_service.cc:1467] Processing CreateTablet for tablet 43a21f53ff1c48a8a7a8baca8011695a (DEFAULT_TABLE table=test-table [id=0a77262a3e284fe7a11b2c5edbb5ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:55.765003 24364 tablet_service.cc:1467] Processing CreateTablet for tablet 43a21f53ff1c48a8a7a8baca8011695a (DEFAULT_TABLE table=test-table [id=0a77262a3e284fe7a11b2c5edbb5ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:15:55.765162 24093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a21f53ff1c48a8a7a8baca8011695a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:55.765177 24229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a21f53ff1c48a8a7a8baca8011695a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:55.765337 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43a21f53ff1c48a8a7a8baca8011695a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:55.768285 24450 tablet_bootstrap.cc:492] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Bootstrap starting.
I20260812 05:15:55.768781 24451 tablet_bootstrap.cc:492] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c: Bootstrap starting.
I20260812 05:15:55.769032 24452 tablet_bootstrap.cc:492] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Bootstrap starting.
I20260812 05:15:55.769238 24450 tablet_bootstrap.cc:654] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:55.769577 24451 tablet_bootstrap.cc:654] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:55.769763 24452 tablet_bootstrap.cc:654] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:55.769942 24451 log.cc:824] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:55.770020 24450 log.cc:824] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:55.770076 24452 log.cc:824] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:55.770846 24451 tablet_bootstrap.cc:492] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c: No bootstrap required, opened a new log
I20260812 05:15:55.770872 24450 tablet_bootstrap.cc:492] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: No bootstrap required, opened a new log
I20260812 05:15:55.770908 24451 ts_tablet_manager.cc:1397] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:15:55.770939 24450 ts_tablet_manager.cc:1397] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 05:15:55.771203 24452 tablet_bootstrap.cc:492] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: No bootstrap required, opened a new log
I20260812 05:15:55.771281 24452 ts_tablet_manager.cc:1397] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:15:55.772960 24451 raft_consensus.cc:348] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.773159 24451 raft_consensus.cc:374] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:55.773209 24451 raft_consensus.cc:729] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08396e417cd049b180da69bdb458a30c, State: Initialized, Role: FOLLOWER
I20260812 05:15:55.773348 24451 consensus_queue.cc:260] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.773723 24451 ts_tablet_manager.cc:1428] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 05:15:55.773802 24160 heartbeater.cc:503] Master 127.18.166.254:38199 was elected leader, sending a full tablet report...
I20260812 05:15:55.774255 24452 raft_consensus.cc:348] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.774256 24450 raft_consensus.cc:348] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.774441 24452 raft_consensus.cc:374] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:55.774441 24450 raft_consensus.cc:374] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:55.774479 24450 raft_consensus.cc:729] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d88cedf7321745f2bfa33dbb69a78fe3, State: Initialized, Role: FOLLOWER
I20260812 05:15:55.774479 24452 raft_consensus.cc:729] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31dfac2d8bf1449292984b3d4228f9c0, State: Initialized, Role: FOLLOWER
I20260812 05:15:55.774624 24452 consensus_queue.cc:260] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.774669 24450 consensus_queue.cc:260] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.774950 24450 ts_tablet_manager.cc:1428] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 05:15:55.774950 24452 ts_tablet_manager.cc:1428] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 05:15:55.775148 24435 heartbeater.cc:503] Master 127.18.166.254:38199 was elected leader, sending a full tablet report...
I20260812 05:15:55.775534 24297 heartbeater.cc:503] Master 127.18.166.254:38199 was elected leader, sending a full tablet report...
I20260812 05:15:55.781008 24459 raft_consensus.cc:482] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:15:55.781145 24459 raft_consensus.cc:504] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.781656 24459 leader_election.cc:288] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08396e417cd049b180da69bdb458a30c (127.18.166.193:42295), 31dfac2d8bf1449292984b3d4228f9c0 (127.18.166.195:43607)
I20260812 05:15:55.783308 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.781856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:42295 (local address 127.18.166.194:37925)
0812 05:15:55.782163 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.782173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.782239 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.782381 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.782381 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.782394 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.782933 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.782934 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.782941 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.782955 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.783147 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.783147 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.783148 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.783198 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.783208 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":105,"thread_start_us":117,"threads_started":1}
I20260812 05:15:55.783308 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.781986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:37925 (local address 127.18.166.193:42295)
0812 05:15:55.782047 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.782049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.782251 (+ 202us) server_negotiation.cc:407] Connection header received
0812 05:15:55.782304 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.782305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.782311 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.782376 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.783036 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.783037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.783038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.783043 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.783061 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.783066 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.783121 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.783210 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.783226 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:15:55.783347 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.781919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:46085 (local address 127.18.166.195:43607)
0812 05:15:55.782069 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.782071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.782081 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:15:55.782114 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.782116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.782121 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.782189 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.782914 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.782916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.782917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.782922 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.782941 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.782946 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.782999 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.783239 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.783253 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260812 05:15:55.783210 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.781848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43607 (local address 127.18.166.194:46085)
0812 05:15:55.781914 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.781928 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:55.781973 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:55.782209 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:55.782210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:55.782212 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:55.782780 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:15:55.782780 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:55.782788 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:15:55.782800 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:55.783048 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:55.783049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:55.783050 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:15:55.783102 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:15:55.783118 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:15:55.783727 24113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a21f53ff1c48a8a7a8baca8011695a" candidate_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08396e417cd049b180da69bdb458a30c" is_pre_election: true
I20260812 05:15:55.783913 24113 raft_consensus.cc:2454] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d88cedf7321745f2bfa33dbb69a78fe3 in term 0.
I20260812 05:15:55.784206 24182 leader_election.cc:302] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08396e417cd049b180da69bdb458a30c, d88cedf7321745f2bfa33dbb69a78fe3; no voters:
I20260812 05:15:55.784374 24459 raft_consensus.cc:2781] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:15:55.784350 24385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a21f53ff1c48a8a7a8baca8011695a" candidate_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31dfac2d8bf1449292984b3d4228f9c0" is_pre_election: true
I20260812 05:15:55.784444 24459 raft_consensus.cc:482] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:15:55.784492 24459 raft_consensus.cc:3037] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.784508 24385 raft_consensus.cc:2454] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d88cedf7321745f2bfa33dbb69a78fe3 in term 0.
I20260812 05:15:55.785328 24459 raft_consensus.cc:504] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.785530 24459 leader_election.cc:288] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [CANDIDATE]: Term 1 election: Requested vote from peers 08396e417cd049b180da69bdb458a30c (127.18.166.193:42295), 31dfac2d8bf1449292984b3d4228f9c0 (127.18.166.195:43607)
I20260812 05:15:55.785722 24113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a21f53ff1c48a8a7a8baca8011695a" candidate_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08396e417cd049b180da69bdb458a30c"
I20260812 05:15:55.785722 24385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43a21f53ff1c48a8a7a8baca8011695a" candidate_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31dfac2d8bf1449292984b3d4228f9c0"
I20260812 05:15:55.785816 24385 raft_consensus.cc:3037] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.785818 24113 raft_consensus.cc:3037] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:55.786830 24113 raft_consensus.cc:2454] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d88cedf7321745f2bfa33dbb69a78fe3 in term 1.
I20260812 05:15:55.786830 24385 raft_consensus.cc:2454] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d88cedf7321745f2bfa33dbb69a78fe3 in term 1.
I20260812 05:15:55.787025 24182 leader_election.cc:302] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08396e417cd049b180da69bdb458a30c, d88cedf7321745f2bfa33dbb69a78fe3; no voters:
I20260812 05:15:55.787147 24459 raft_consensus.cc:2781] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:55.787319 24459 raft_consensus.cc:686] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 1 LEADER]: Becoming Leader. State: Replica: d88cedf7321745f2bfa33dbb69a78fe3, State: Running, Role: LEADER
I20260812 05:15:55.787456 24459 consensus_queue.cc:237] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } }
I20260812 05:15:55.788445 23961 catalog_manager.cc:5495] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d88cedf7321745f2bfa33dbb69a78fe3 (127.18.166.194). New cstate: current_term: 1 leader_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d88cedf7321745f2bfa33dbb69a78fe3" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 } health_report { overall_health: UNKNOWN } } }
I20260812 05:15:55.792459 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.791471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.18.166.194:35965)
0812 05:15:55.791517 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:55.791519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:55.791527 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:15:55.791614 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:55.791615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:55.791628 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:55.791703 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:55.792214 (+ 511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:55.792215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:55.792215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:55.792219 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:55.792235 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:55.792239 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:55.792285 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:55.792316 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 05:15:55.792328 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:15:55.794661 24113 raft_consensus.cc:1261] T 43a21f53ff1c48a8a7a8baca8011695a P 08396e417cd049b180da69bdb458a30c [term 1 FOLLOWER]: Refusing update from remote peer d88cedf7321745f2bfa33dbb69a78fe3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:55.794661 24385 raft_consensus.cc:1261] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 1 FOLLOWER]: Refusing update from remote peer d88cedf7321745f2bfa33dbb69a78fe3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:15:55.795091 24461 consensus_queue.cc:1035] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31dfac2d8bf1449292984b3d4228f9c0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:55.795265 24459 consensus_queue.cc:1035] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08396e417cd049b180da69bdb458a30c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:15:55.798207 24464 mvcc.cc:205] Tried to move back new op lower bound from 7317552151731499008 to 7317552151706386432. Current Snapshot: MvccSnapshot[applied={T|T < 7317552151731499008}]
I20260812 05:15:55.798718 24465 mvcc.cc:205] Tried to move back new op lower bound from 7317552151731499008 to 7317552151706386432. Current Snapshot: MvccSnapshot[applied={T|T < 7317552151731499008}]
I20260812 05:15:55.799372 24466 mvcc.cc:205] Tried to move back new op lower bound from 7317552151731499008 to 7317552151706386432. Current Snapshot: MvccSnapshot[applied={T|T < 7317552151731499008 or (T in {7317552151731499008})}]
I20260812 05:15:55.801007 23961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:55.801141 23961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:15:55.802119 23961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8a4db377d2347428f1511a3e0fd7129: Sending DeleteTablet for 3 replicas of tablet 43a21f53ff1c48a8a7a8baca8011695a
I20260812 05:15:55.802520 24229 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a21f53ff1c48a8a7a8baca8011695a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:55 UTC) from {username='slave'} at 127.0.0.1:50654
I20260812 05:15:55.802575 24364 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a21f53ff1c48a8a7a8baca8011695a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:55 UTC) from {username='slave'} at 127.0.0.1:56624
I20260812 05:15:55.802839 24476 tablet_replica.cc:330] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: stopping tablet replica
I20260812 05:15:55.802850 24093 tablet_service.cc:1514] Processing DeleteTablet for tablet 43a21f53ff1c48a8a7a8baca8011695a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:15:55 UTC) from {username='slave'} at 127.0.0.1:45412
I20260812 05:15:55.802960 24476 raft_consensus.cc:2229] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:15:55.803108 24476 raft_consensus.cc:2258] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:55.803292 24475 tablet_replica.cc:330] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: stopping tablet replica
I20260812 05:15:55.803409 24475 raft_consensus.cc:2229] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:15:55.803467 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24026
I20260812 05:15:55.803524 24475 raft_consensus.cc:2258] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:15:55.806187 24476 ts_tablet_manager.cc:1905] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:55.808189 24476 ts_tablet_manager.cc:1918] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:15:55.808295 24476 log.cc:1196] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/43a21f53ff1c48a8a7a8baca8011695a
I20260812 05:15:55.808593 24476 ts_tablet_manager.cc:1939] T 43a21f53ff1c48a8a7a8baca8011695a P d88cedf7321745f2bfa33dbb69a78fe3: Deleting consensus metadata
I20260812 05:15:55.809301 24475 ts_tablet_manager.cc:1905] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:15:55.811172 23948 catalog_manager.cc:4841] TS d88cedf7321745f2bfa33dbb69a78fe3 (127.18.166.194:35965): tablet 43a21f53ff1c48a8a7a8baca8011695a (table test-table [id=0a77262a3e284fe7a11b2c5edbb5ab42]) successfully deleted
I20260812 05:15:55.811231 24475 ts_tablet_manager.cc:1918] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:15:55.811321 24475 log.cc:1196] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/43a21f53ff1c48a8a7a8baca8011695a
I20260812 05:15:55.811628 24475 ts_tablet_manager.cc:1939] T 43a21f53ff1c48a8a7a8baca8011695a P 31dfac2d8bf1449292984b3d4228f9c0: Deleting consensus metadata
I20260812 05:15:55.812125 23946 catalog_manager.cc:4841] TS 31dfac2d8bf1449292984b3d4228f9c0 (127.18.166.195:43607): tablet 43a21f53ff1c48a8a7a8baca8011695a (table test-table [id=0a77262a3e284fe7a11b2c5edbb5ab42]) successfully deleted
W20260812 05:15:55.815413 23947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:42295 (error 108)
I20260812 05:15:55.815685 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24163
I20260812 05:15:55.816989 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:55.816845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:42295 (local address 127.0.0.1:45414)
0812 05:15:55.816888 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:55.816943 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:42295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 05:15:55.817330 23947 catalog_manager.cc:4584] TS 08396e417cd049b180da69bdb458a30c (127.18.166.193:42295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43a21f53ff1c48a8a7a8baca8011695a: Network error: Client connection negotiation failed: client connection to 127.18.166.193:42295: connect: Connection refused (error 111)
I20260812 05:15:55.823683 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24300
I20260812 05:15:55.829864 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 23929
2026-08-12T05:15:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (661 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 05:15:55.851769 19099 mini_oidc.cc:152] Starting JWKS server
I20260812 05:15:55.853034 19099 webserver.cc:460] Webserver started at https://127.0.0.1:32991/ using document root <none> and password file <none>
I20260812 05:15:55.853166 19099 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 05:15:55.853370 19099 webserver.cc:460] Webserver started at http://127.0.0.1:39125/ using document root <none> and password file <none>
2026-08-12T05:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:55Z Disabled control of system clock
I20260812 05:15:55.861933 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41101
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:41101
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32991/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/testchainca.cert with env {}
W20260812 05:15:55.949183 24484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:55.949510 24484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:55.949548 24484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:55.951577 24484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:55.951650 24484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:55.951669 24484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:55.951709 24484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:55.951743 24484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:15:55.951764 24484 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:15:55.951792 24484 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32991/jwks.json
I20260812 05:15:55.954262 24484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:41101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32991/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:55.954583 24484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:55.954833 24484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:55.959944 24489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.960072 24484 server_base.cc:1034] running on GCE node
W20260812 05:15:55.959925 24493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:55.960188 24491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:55.960548 24484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:55.960884 24484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:55.962062 24484 hybrid_clock.cc:648] HybridClock initialized: now 1786511755962040 us; error 42 us; skew 500 ppm
I20260812 05:15:55.963605 24484 webserver.cc:460] Webserver started at http://127.18.166.254:38517/ using document root <none> and password file <none>
I20260812 05:15:55.963928 24484 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:55.964012 24484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:55.964213 24484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:55.965734 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "a88b0cd4e1a844189ca48d76ec0c41f3"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "4408f75537fc2bc8e0506f7772472b85"
server_key_iv: "b1c0eeeb27deaf41ca98feec84333e82"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.966172 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "a88b0cd4e1a844189ca48d76ec0c41f3"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "4408f75537fc2bc8e0506f7772472b85"
server_key_iv: "b1c0eeeb27deaf41ca98feec84333e82"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.971155 24484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260812 05:15:55.972239 24500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:55.972467 24484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:55.972606 24484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "a88b0cd4e1a844189ca48d76ec0c41f3"
format_stamp: "Formatted at 2026-08-12 05:15:55 on dist-test-slave-srgv"
server_key: "4408f75537fc2bc8e0506f7772472b85"
server_key_iv: "b1c0eeeb27deaf41ca98feec84333e82"
server_key_version: "encryptionkey@0"
I20260812 05:15:55.972710 24484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:55.995895 24484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:55.996305 24484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:55.996486 24484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:56.000931 24484 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:41101
I20260812 05:15:56.001055 24553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:41101 every 8 connection(s)
I20260812 05:15:56.001452 24484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:56.002655 24554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:56.005235 24554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3: Bootstrap starting.
I20260812 05:15:56.005941 24554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:56.006218 24554 log.cc:824] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:56.006947 24554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3: No bootstrap required, opened a new log
I20260812 05:15:56.008664 24554 raft_consensus.cc:348] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41101 } }
I20260812 05:15:56.008829 24554 raft_consensus.cc:374] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:56.008852 24554 raft_consensus.cc:729] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88b0cd4e1a844189ca48d76ec0c41f3, State: Initialized, Role: FOLLOWER
I20260812 05:15:56.009011 24554 consensus_queue.cc:260] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41101 } }
I20260812 05:15:56.009101 24554 raft_consensus.cc:388] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:56.009127 24554 raft_consensus.cc:482] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:56.009169 24554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:56.009776 24554 raft_consensus.cc:504] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41101 } }
I20260812 05:15:56.009872 24554 leader_election.cc:302] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a88b0cd4e1a844189ca48d76ec0c41f3; no voters:
I20260812 05:15:56.010028 24554 leader_election.cc:288] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:56.010124 24557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:56.010300 24557 raft_consensus.cc:686] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [term 1 LEADER]: Becoming Leader. State: Replica: a88b0cd4e1a844189ca48d76ec0c41f3, State: Running, Role: LEADER
I20260812 05:15:56.010329 24554 sys_catalog.cc:564] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:56.010407 24557 consensus_queue.cc:237] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41101 } }
I20260812 05:15:56.010831 24558 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41101 } } }
I20260812 05:15:56.010951 24558 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.011183 24559 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a88b0cd4e1a844189ca48d76ec0c41f3. Latest consensus state: current_term: 1 leader_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88b0cd4e1a844189ca48d76ec0c41f3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41101 } } }
I20260812 05:15:56.011286 24559 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.011308 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24484
I20260812 05:15:56.011420 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:56.011644 19099 external_mini_cluster.cc:1428] Setting key 6e22dd7f1dd601e2ca7a455d586d01af
I20260812 05:15:56.011901 24566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:56.012960 24566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:56.014798 24566 catalog_manager.cc:1347] Generated new cluster ID: b3aab2b569af41d39c6b2a52e26a61a9
I20260812 05:15:56.014864 24566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:56.017617 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.012273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.18.166.254:41101)
0812 05:15:56.012509 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.012519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.012562 (+ 43us) server_negotiation.cc:407] Connection header received
0812 05:15:56.012670 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.012676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.012736 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.012890 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:56.013126 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.014492 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.015455 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.015914 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.016059 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.016849 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:56.016853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:56.016853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:56.017047 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:56.017082 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:56.017095 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:56.017174 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:56.017259 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 05:15:56.017349 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260812 05:15:56.018653 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:15:56.050190 24566 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:56.051009 24566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:56.059887 24566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a88b0cd4e1a844189ca48d76ec0c41f3: Generated new TSK 0
I20260812 05:15:56.060148 24566 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 05:15:56.087720 24586 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 05:15:56.087916 24586 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 05:15:56.088106 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.070795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33524 (local address 127.18.166.254:41101)
0812 05:15:56.070869 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.070871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.071025 (+ 154us) server_negotiation.cc:407] Connection header received
0812 05:15:56.071117 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.071118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.071132 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.071230 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:15:56.071400 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.072176 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.073040 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.073349 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.073473 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.073780 (+ 307us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:15:56.087828 (+ 14048us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32991/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 05:15:56.088024 (+ 196us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33524: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32991/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 05:15:56.088187 24576 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33524: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32991/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 05:15:56.088896 24585 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:56.070678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41101 (local address 127.0.0.1:33524)
0812 05:15:56.070947 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:56.070958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:56.071027 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:56.071247 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:56.071248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:56.071250 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:15:56.071326 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.071328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.072313 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.072313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.072934 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.072936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.073536 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.073537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.073598 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.073628 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:15:56.088167 (+ 14539us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32991/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 05:15:56.088494 (+ 327us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.166.254:41101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32991/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":151,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.166.254:41101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32991/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 05:15:56.089679 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24484
2026-08-12T05:15:56Z chronyd exiting
I20260812 05:15:56.127794 19099 test_util.cc:175] -----------------------------------------------
I20260812 05:15:56.127923 19099 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786511731160932-19099-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (277 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 05:15:56.128799 19099 mini_oidc.cc:152] Starting JWKS server
I20260812 05:15:56.129645 19099 webserver.cc:460] Webserver started at https://127.0.0.1:40685/ using document root <none> and password file <none>
I20260812 05:15:56.129726 19099 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 05:15:56.129915 19099 webserver.cc:460] Webserver started at http://127.0.0.1:45853/ using document root <none> and password file <none>
2026-08-12T05:15:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:56Z Disabled control of system clock
I20260812 05:15:56.138891 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35307
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:46645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35307
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40685/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/testchainca.cert with env {}
W20260812 05:15:56.228003 24593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:56.228246 24593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:56.228279 24593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:56.229660 24593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:56.229707 24593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:56.229722 24593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:56.229734 24593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:56.229784 24593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:15:56.229801 24593 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:15:56.229825 24593 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40685/jwks.json
I20260812 05:15:56.231745 24593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40685/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:56.232023 24593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:56.232242 24593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:56.237193 24598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:56.237211 24593 server_base.cc:1034] running on GCE node
W20260812 05:15:56.237211 24599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:56.237375 24601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:56.237711 24593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:56.238029 24593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:56.239226 24593 hybrid_clock.cc:648] HybridClock initialized: now 1786511756239207 us; error 29 us; skew 500 ppm
I20260812 05:15:56.240590 24593 webserver.cc:460] Webserver started at http://127.18.166.254:41941/ using document root <none> and password file <none>
I20260812 05:15:56.240911 24593 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:56.240999 24593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:56.241178 24593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:56.242435 24593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "23bd111964c443f6bcb2e4499dc18bf9"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "61b628d8fdf6b4c7a5a0051906ee57e6"
server_key_iv: "0a34ae55b0351220763965e9394ce41c"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.242871 24593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "23bd111964c443f6bcb2e4499dc18bf9"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "61b628d8fdf6b4c7a5a0051906ee57e6"
server_key_iv: "0a34ae55b0351220763965e9394ce41c"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.244436 24593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:15:56.245340 24607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:56.245582 24593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:56.245674 24593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "23bd111964c443f6bcb2e4499dc18bf9"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "61b628d8fdf6b4c7a5a0051906ee57e6"
server_key_iv: "0a34ae55b0351220763965e9394ce41c"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.245766 24593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:56.258927 24593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:56.259341 24593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:56.259533 24593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:56.264024 24593 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:35307
I20260812 05:15:56.264134 24662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:35307 every 8 connection(s)
I20260812 05:15:56.264525 24593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:56.265719 24663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:56.268278 24663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9: Bootstrap starting.
I20260812 05:15:56.268977 24663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:56.269294 24663 log.cc:824] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:56.270118 24663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9: No bootstrap required, opened a new log
I20260812 05:15:56.271896 24663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23bd111964c443f6bcb2e4499dc18bf9" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35307 } }
I20260812 05:15:56.272068 24663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:56.272094 24663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23bd111964c443f6bcb2e4499dc18bf9, State: Initialized, Role: FOLLOWER
I20260812 05:15:56.272323 24663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23bd111964c443f6bcb2e4499dc18bf9" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35307 } }
I20260812 05:15:56.272416 24663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:56.272470 24663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:56.272528 24663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:56.273202 24663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23bd111964c443f6bcb2e4499dc18bf9" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35307 } }
I20260812 05:15:56.273342 24663 leader_election.cc:302] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23bd111964c443f6bcb2e4499dc18bf9; no voters:
I20260812 05:15:56.273590 24663 leader_election.cc:288] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:56.273674 24666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:56.273872 24666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [term 1 LEADER]: Becoming Leader. State: Replica: 23bd111964c443f6bcb2e4499dc18bf9, State: Running, Role: LEADER
I20260812 05:15:56.273891 24663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:56.273978 24666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23bd111964c443f6bcb2e4499dc18bf9" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35307 } }
I20260812 05:15:56.274386 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24593
I20260812 05:15:56.274374 24668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23bd111964c443f6bcb2e4499dc18bf9. Latest consensus state: current_term: 1 leader_uuid: "23bd111964c443f6bcb2e4499dc18bf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23bd111964c443f6bcb2e4499dc18bf9" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35307 } } }
I20260812 05:15:56.274456 24668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.274499 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:56.274756 19099 external_mini_cluster.cc:1428] Setting key 4b9c02f2d7dc9eed8f8a2f332cc47dcc
I20260812 05:15:56.274768 24667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23bd111964c443f6bcb2e4499dc18bf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23bd111964c443f6bcb2e4499dc18bf9" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35307 } } }
I20260812 05:15:56.274847 24667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.275267 24673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:56.275842 24673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:56.277594 24673 catalog_manager.cc:1347] Generated new cluster ID: 7b90139645b3451ab4059153471f7c5b
I20260812 05:15:56.277659 24673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:56.281904 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.275465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.18.166.254:35307)
0812 05:15:56.275695 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.275707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.275744 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:15:56.275877 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.275883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.275959 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.276116 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:56.276379 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.277822 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.279043 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.279794 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.279961 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.280810 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:56.280815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:56.280816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:56.281084 (+ 268us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:56.281137 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:56.281155 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:56.281237 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:56.281318 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 05:15:56.281613 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260812 05:15:56.283092 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:15:56.295673 24673 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:56.296254 24673 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:56.308876 24673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23bd111964c443f6bcb2e4499dc18bf9: Generated new TSK 0
I20260812 05:15:56.309150 24673 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 05:15:56.336349 24693 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 05:15:56.336465 24693 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 05:15:56.336854 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.320876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.18.166.254:35307)
0812 05:15:56.320955 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.320957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.320995 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:15:56.321079 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.321080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.321095 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.321202 (+ 107us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:15:56.321337 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.322208 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.323080 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.323419 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.323500 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.323705 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:15:56.336579 (+ 12874us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40685/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 05:15:56.336769 (+ 190us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33700: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40685/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 05:15:56.336921 24684 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33700: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40685/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 05:15:56.336901 24692 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:56.320714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35307 (local address 127.0.0.1:33700)
0812 05:15:56.320929 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:56.320946 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:56.321003 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:56.321198 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:56.321199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:56.321200 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:15:56.321260 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.321262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.322364 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.322365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.322969 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.322971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.323511 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.323512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.323554 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.323560 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:15:56.336718 (+ 13158us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40685/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 05:15:56.336828 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.166.254:35307: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40685/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":115,"threads_started":1}
I20260812 05:15:56.337230 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24593
2026-08-12T05:15:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 05:15:56.377765 19099 mini_oidc.cc:152] Starting JWKS server
I20260812 05:15:56.378573 19099 webserver.cc:460] Webserver started at https://127.0.0.1:36839/ using document root <none> and password file <none>
I20260812 05:15:56.378660 19099 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 05:15:56.378822 19099 webserver.cc:460] Webserver started at http://127.0.0.1:43123/ using document root <none> and password file <none>
2026-08-12T05:15:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:56Z Disabled control of system clock
I20260812 05:15:56.389034 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:32837
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:42489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:32837
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36839/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 05:15:56.479600 24700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:56.479863 24700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:56.479890 24700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:56.481244 24700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:56.481300 24700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:56.481314 24700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:56.481328 24700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:56.481339 24700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:15:56.481352 24700 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:15:56.481364 24700 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36839/jwks.json
I20260812 05:15:56.483100 24700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:32837
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36839/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:56.483373 24700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:56.483666 24700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:56.489076 24708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:56.489094 24705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:56.489104 24706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:56.489503 24700 server_base.cc:1034] running on GCE node
I20260812 05:15:56.489746 24700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:56.490021 24700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:56.491212 24700 hybrid_clock.cc:648] HybridClock initialized: now 1786511756491200 us; error 43 us; skew 500 ppm
I20260812 05:15:56.492643 24700 webserver.cc:460] Webserver started at http://127.18.166.254:38327/ using document root <none> and password file <none>
I20260812 05:15:56.492921 24700 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:56.492998 24700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:56.493165 24700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:56.494498 24700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "2ef597b787bd47bd9841f23bd796bce1"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "8da99e394442526b3afd1e1aaa015c8a"
server_key_iv: "7e7f3704b9cb7d82188cb61fe0ad6381"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.494899 24700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "2ef597b787bd47bd9841f23bd796bce1"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "8da99e394442526b3afd1e1aaa015c8a"
server_key_iv: "7e7f3704b9cb7d82188cb61fe0ad6381"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.496356 24700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:15:56.497260 24714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:56.497498 24700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:56.497603 24700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "2ef597b787bd47bd9841f23bd796bce1"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "8da99e394442526b3afd1e1aaa015c8a"
server_key_iv: "7e7f3704b9cb7d82188cb61fe0ad6381"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.497691 24700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:56.516320 24700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:56.516741 24700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:56.516933 24700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:56.521607 24700 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:32837
I20260812 05:15:56.521646 24771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:32837 every 8 connection(s)
I20260812 05:15:56.522074 24700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:56.522929 24772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:56.524919 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24700
I20260812 05:15:56.525043 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:56.525255 19099 external_mini_cluster.cc:1428] Setting key a783b4136e68784110d73430802b76a0
I20260812 05:15:56.527215 24772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1: Bootstrap starting.
I20260812 05:15:56.528081 24772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:56.528460 24772 log.cc:824] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:56.529556 24772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1: No bootstrap required, opened a new log
I20260812 05:15:56.532249 24772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef597b787bd47bd9841f23bd796bce1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 32837 } }
I20260812 05:15:56.532482 24772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:56.532508 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.525884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.18.166.254:32837)
0812 05:15:56.526153 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.526164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.526201 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:15:56.526835 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.526845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.526946 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.527128 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:56.527469 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.528905 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.529952 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.530614 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.530782 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.531631 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:56.531638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:56.531638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:56.531930 (+ 292us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:56.531979 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:56.531996 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:56.532078 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:56.532140 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 05:15:56.532239 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":105,"threads_started":1}
I20260812 05:15:56.532526 24772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef597b787bd47bd9841f23bd796bce1, State: Initialized, Role: FOLLOWER
I20260812 05:15:56.532696 24772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef597b787bd47bd9841f23bd796bce1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 32837 } }
I20260812 05:15:56.532799 24772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:56.532835 24772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:56.532879 24772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:56.533742 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:15:56.533902 24772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef597b787bd47bd9841f23bd796bce1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 32837 } }
I20260812 05:15:56.534018 24772 leader_election.cc:302] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ef597b787bd47bd9841f23bd796bce1; no voters:
I20260812 05:15:56.534225 24772 leader_election.cc:288] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:56.534297 24777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:56.534472 24777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [term 1 LEADER]: Becoming Leader. State: Replica: 2ef597b787bd47bd9841f23bd796bce1, State: Running, Role: LEADER
I20260812 05:15:56.534523 24772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:56.534739 24777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef597b787bd47bd9841f23bd796bce1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 32837 } }
I20260812 05:15:56.535120 24777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ef597b787bd47bd9841f23bd796bce1. Latest consensus state: current_term: 1 leader_uuid: "2ef597b787bd47bd9841f23bd796bce1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef597b787bd47bd9841f23bd796bce1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 32837 } } }
I20260812 05:15:56.535193 24777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.535362 24778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ef597b787bd47bd9841f23bd796bce1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ef597b787bd47bd9841f23bd796bce1" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 32837 } } }
I20260812 05:15:56.535460 24778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.536046 24783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:56.536520 24783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:56.538686 24783 catalog_manager.cc:1347] Generated new cluster ID: 797be67e29a74495a9b2a0e72d29d06c
I20260812 05:15:56.538765 24783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:56.558180 24783 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:56.559027 24783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:56.563301 24783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ef597b787bd47bd9841f23bd796bce1: Generated new TSK 0
I20260812 05:15:56.563576 24783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:15:56.583063 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.578318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.18.166.254:32837)
0812 05:15:56.578399 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.578402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.578544 (+ 142us) server_negotiation.cc:407] Connection header received
0812 05:15:56.578647 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.578648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.578660 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.578734 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:15:56.578953 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.579515 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.580730 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.580961 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.581034 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.581268 (+ 234us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:15:56.582775 (+ 1507us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36839/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 05:15:56.582958 (+ 183us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33742: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36839/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 05:15:56.583125 24803 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:56.578243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:32837 (local address 127.0.0.1:33742)
0812 05:15:56.578451 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:56.578465 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:56.578552 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:56.578757 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:56.578758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:56.578760 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:15:56.578849 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.578851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.579625 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.579626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.580598 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.580602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.581065 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.581067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.581148 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.581155 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:15:56.582901 (+ 1746us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36839/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 05:15:56.583015 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.166.254:32837: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36839/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":130,"threads_started":1}
W20260812 05:15:56.583426 24775 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33742: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36839/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260812 05:15:56.583817 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24700
E20260812 05:15:56.597535 24804 webserver.cc:533] Webserver: sslize failed:
E20260812 05:15:56.597656 24804 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T05:15:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T05:15:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:56Z Disabled control of system clock
I20260812 05:15:56.635607 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42253
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:42253
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/testchainca.cert with env {}
W20260812 05:15:56.727393 24808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:56.727638 24808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:56.727662 24808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:56.728904 24808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:56.728955 24808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:56.728966 24808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:56.728979 24808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:56.728992 24808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:15:56.729004 24808 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:15:56.729017 24808 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 05:15:56.730819 24808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:42253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:56.731130 24808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:56.731364 24808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:56.737133 24816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:56.737133 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:56.737388 24808 server_base.cc:1034] running on GCE node
W20260812 05:15:56.737136 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:56.737735 24808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:56.738018 24808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:56.739176 24808 hybrid_clock.cc:648] HybridClock initialized: now 1786511756739150 us; error 51 us; skew 500 ppm
I20260812 05:15:56.740684 24808 webserver.cc:460] Webserver started at http://127.18.166.254:35507/ using document root <none> and password file <none>
I20260812 05:15:56.740973 24808 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:56.741026 24808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:56.741164 24808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:56.742422 24808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "1d2d98774fdc43098d1cf439c4c18247"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "c3de087a55a286f6eb2d17deda0808b2"
server_key_iv: "c72eb5d55523c607655b12c4cad9d97e"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.742844 24808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "1d2d98774fdc43098d1cf439c4c18247"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "c3de087a55a286f6eb2d17deda0808b2"
server_key_iv: "c72eb5d55523c607655b12c4cad9d97e"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.744424 24808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:15:56.745312 24822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:56.745599 24808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:56.745699 24808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "1d2d98774fdc43098d1cf439c4c18247"
format_stamp: "Formatted at 2026-08-12 05:15:56 on dist-test-slave-srgv"
server_key: "c3de087a55a286f6eb2d17deda0808b2"
server_key_iv: "c72eb5d55523c607655b12c4cad9d97e"
server_key_version: "encryptionkey@0"
I20260812 05:15:56.745766 24808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:56.757644 24808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:56.757937 24808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:56.758044 24808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:56.762297 24808 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:42253
I20260812 05:15:56.762398 24874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:42253 every 8 connection(s)
I20260812 05:15:56.762763 24808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:56.763907 24875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:56.766594 24875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247: Bootstrap starting.
I20260812 05:15:56.767297 24875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:56.767621 24875 log.cc:824] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:56.768461 24875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247: No bootstrap required, opened a new log
I20260812 05:15:56.770370 24875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d98774fdc43098d1cf439c4c18247" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42253 } }
I20260812 05:15:56.770577 24875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:56.770633 24875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2d98774fdc43098d1cf439c4c18247, State: Initialized, Role: FOLLOWER
I20260812 05:15:56.770826 24875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d98774fdc43098d1cf439c4c18247" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42253 } }
I20260812 05:15:56.770920 24875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:56.770965 24875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:56.771015 24875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:56.771687 24875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d98774fdc43098d1cf439c4c18247" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42253 } }
I20260812 05:15:56.771822 24875 leader_election.cc:302] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d2d98774fdc43098d1cf439c4c18247; no voters:
I20260812 05:15:56.772028 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24808
I20260812 05:15:56.772060 24875 leader_election.cc:288] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:56.772091 24878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:56.772156 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:56.772221 24878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [term 1 LEADER]: Becoming Leader. State: Replica: 1d2d98774fdc43098d1cf439c4c18247, State: Running, Role: LEADER
I20260812 05:15:56.772325 24878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d98774fdc43098d1cf439c4c18247" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42253 } }
I20260812 05:15:56.772372 24875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:56.772394 19099 external_mini_cluster.cc:1428] Setting key e9f422507f88acdcc1073df4f0222298
I20260812 05:15:56.772682 24878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d2d98774fdc43098d1cf439c4c18247. Latest consensus state: current_term: 1 leader_uuid: "1d2d98774fdc43098d1cf439c4c18247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d98774fdc43098d1cf439c4c18247" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42253 } } }
I20260812 05:15:56.772724 24880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d2d98774fdc43098d1cf439c4c18247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d98774fdc43098d1cf439c4c18247" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42253 } } }
I20260812 05:15:56.772763 24878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.772927 24880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:56.773187 24888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:56.774118 24888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:56.775830 24888 catalog_manager.cc:1347] Generated new cluster ID: f2e7ad91f8fb4291ad71d03aa4c38429
I20260812 05:15:56.775893 24888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:15:56.779588 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.773133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.18.166.254:42253)
0812 05:15:56.773549 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.773559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.773594 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:15:56.773858 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.773866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.773947 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.774101 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:56.774460 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.775882 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.777075 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.777750 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.777892 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.778740 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:56.778746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:56.778746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:15:56.778982 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:56.779025 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:15:56.779041 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:56.779112 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:56.779219 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 05:15:56.779312 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":90,"threads_started":1}
I20260812 05:15:56.780859 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:15:56.796916 24888 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:56.797601 24888 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:56.809784 24888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d2d98774fdc43098d1cf439c4c18247: Generated new TSK 0
I20260812 05:15:56.810057 24888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:15:56.849162 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:56.826361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.18.166.254:42253)
0812 05:15:56.826417 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:56.826419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:56.826523 (+ 104us) server_negotiation.cc:407] Connection header received
0812 05:15:56.826629 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:56.826630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:56.826642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:56.826716 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:15:56.826895 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.827451 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.828772 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.828998 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.829067 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.829325 (+ 258us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:15:56.848907 (+ 19582us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 05:15:56.849086 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57704: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 05:15:56.849241 24890 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57704: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 05:15:56.849362 24905 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:56.826216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42253 (local address 127.0.0.1:57704)
0812 05:15:56.826421 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:56.826435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:56.826532 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:56.826741 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:56.826743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:56.826745 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:15:56.826819 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.826822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.827582 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.827583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.828630 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:56.828635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:56.829121 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:56.829122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:56.829197 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:56.829205 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:15:56.849086 (+ 19881us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 05:15:56.849248 (+ 162us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.166.254:42253: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":125,"threads_started":1}
I20260812 05:15:56.849737 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24808
2026-08-12T05:15:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (243 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (96 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:57 dist-test-slave-srgv krb5kdc[24918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:57 dist-test-slave-srgv krb5kdc[24918](info): set up 2 sockets
Aug 12 05:15:57 dist-test-slave-srgv krb5kdc[24918](info): commencing operation
krb5kdc: starting...
W20260812 05:15:58.120673 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:15:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:15:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:15:58.252249 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41095
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:41095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:58.341907 24931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:58.342187 24931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:58.342218 24931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:58.343634 24931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:15:58.343695 24931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:58.343711 24931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:58.343726 24931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:15:58.343740 24931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:15:58.345608 24931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:41095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:58.345880 24931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:58.346112 24931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:58.351619 24939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.351630 24936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.351630 24937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:58.351702 24931 server_base.cc:1034] running on GCE node
I20260812 05:15:58.352130 24931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:58.352485 24931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:58.353644 24931 hybrid_clock.cc:648] HybridClock initialized: now 1786511758353614 us; error 44 us; skew 500 ppm
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:58.362089 24931 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:15:58.362703 24931 webserver.cc:460] Webserver started at http://127.18.166.254:41309/ using document root <none> and password file <none>
I20260812 05:15:58.363003 24931 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:58.363056 24931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:58.363250 24931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:58.364488 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "5040f37794914a619a19198d2d08abd4"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "bf9338e84b9a93016ff794f4f00f5582"
server_key_iv: "d3a79572f374b573de05db4b0dbbe358"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.364919 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "5040f37794914a619a19198d2d08abd4"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "bf9338e84b9a93016ff794f4f00f5582"
server_key_iv: "d3a79572f374b573de05db4b0dbbe358"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.366595 24931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 05:15:58.367599 24946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.367831 24931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:58.367918 24931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "5040f37794914a619a19198d2d08abd4"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "bf9338e84b9a93016ff794f4f00f5582"
server_key_iv: "d3a79572f374b573de05db4b0dbbe358"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.367990 24931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:58.385562 24931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:58.388258 24931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:58.388393 24931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:58.393008 24931 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:41095
I20260812 05:15:58.393044 25001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:41095 every 8 connection(s)
I20260812 05:15:58.393544 24931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:15:58.394570 25002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:15:58.397205 25002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4: Bootstrap starting.
I20260812 05:15:58.398010 25002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4: Neither blocks nor log segments found. Creating new log.
I20260812 05:15:58.398355 25002 log.cc:824] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4: Log is configured to *not* fsync() on all Append() calls
I20260812 05:15:58.398429 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 24931
I20260812 05:15:58.398571 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:15:58.398797 19099 external_mini_cluster.cc:1428] Setting key 95b912c261b0b92b45ddbededa257fa8
I20260812 05:15:58.399243 25002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4: No bootstrap required, opened a new log
I20260812 05:15:58.401834 25002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5040f37794914a619a19198d2d08abd4" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41095 } }
I20260812 05:15:58.402019 25002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:15:58.402071 25002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5040f37794914a619a19198d2d08abd4, State: Initialized, Role: FOLLOWER
I20260812 05:15:58.402256 25002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5040f37794914a619a19198d2d08abd4" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41095 } }
I20260812 05:15:58.402359 25002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:15:58.402410 25002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:15:58.402460 25002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:15:58.403152 25002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5040f37794914a619a19198d2d08abd4" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41095 } }
I20260812 05:15:58.403302 25002 leader_election.cc:302] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5040f37794914a619a19198d2d08abd4; no voters:
I20260812 05:15:58.403498 25002 leader_election.cc:288] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:15:58.403577 25007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:15:58.403769 25007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [term 1 LEADER]: Becoming Leader. State: Replica: 5040f37794914a619a19198d2d08abd4, State: Running, Role: LEADER
I20260812 05:15:58.403787 25002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:15:58.403865 25007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5040f37794914a619a19198d2d08abd4" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41095 } }
I20260812 05:15:58.404251 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5040f37794914a619a19198d2d08abd4. Latest consensus state: current_term: 1 leader_uuid: "5040f37794914a619a19198d2d08abd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5040f37794914a619a19198d2d08abd4" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41095 } } }
I20260812 05:15:58.404345 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:58.404560 25014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:15:58.405133 25014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:15:58.404265 25008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5040f37794914a619a19198d2d08abd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5040f37794914a619a19198d2d08abd4" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41095 } } }
I20260812 05:15:58.406364 25008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4 [sys.catalog]: This master's current role is: LEADER
I20260812 05:15:58.406958 25014 catalog_manager.cc:1347] Generated new cluster ID: 9a5f0991399c44fcac69e881c6febd15
I20260812 05:15:58.407013 25014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:58.412613 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.399408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.18.166.254:41095)
0812 05:15:58.399631 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:58.399641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:58.399675 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:15:58.400662 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:58.400670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:58.400728 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:58.400856 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:58.401763 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.403115 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.404307 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.404859 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.405000 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.408221 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:58.408227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:58.408227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:58.408240 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:58.410594 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.411127 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.411131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.411131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.411258 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.411502 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.411503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.411503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.411844 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:58.412009 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:58.412230 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 05:15:58.412386 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":122,"threads_started":1}
I20260812 05:15:58.414901 25014 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:15:58.415699 25014 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:15:58.428843 25014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5040f37794914a619a19198d2d08abd4: Generated new TSK 0
I20260812 05:15:58.429112 25014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:15:58.465588 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41095
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:58.571564 25028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:58.571873 25028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:58.571923 25028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:58.573455 25028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:58.573556 25028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:58.573606 25028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:15:58.575403 25028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:58.575753 25028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:58.576037 25028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:15:58.581606 25028 server_base.cc:1034] running on GCE node
W20260812 05:15:58.581553 25034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.581548 25036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.581553 25033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:58.582001 25028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:58.582304 25028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:58.583465 25028 hybrid_clock.cc:648] HybridClock initialized: now 1786511758583451 us; error 32 us; skew 500 ppm
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:58.586464 25028 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:15:58.587054 25028 webserver.cc:460] Webserver started at http://127.18.166.193:46333/ using document root <none> and password file <none>
I20260812 05:15:58.587339 25028 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:58.587396 25028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:58.587538 25028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:58.588732 25028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "4f73eb0be46a4a159ed01bf75401e4c4"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "bb9c04b540b39bba35aff766aa97f6fd"
server_key_iv: "f05a53d65365dba972cd93a07dc39d2a"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.589141 25028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "4f73eb0be46a4a159ed01bf75401e4c4"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "bb9c04b540b39bba35aff766aa97f6fd"
server_key_iv: "f05a53d65365dba972cd93a07dc39d2a"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.590798 25028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 05:15:58.591691 25044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.591945 25028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:58.592063 25028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "4f73eb0be46a4a159ed01bf75401e4c4"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "bb9c04b540b39bba35aff766aa97f6fd"
server_key_iv: "f05a53d65365dba972cd93a07dc39d2a"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.592155 25028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:58.608736 25028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:58.611155 25028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:58.611356 25028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:58.611608 25028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:58.611944 25028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:58.612001 25028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.612049 25028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:58.612080 25028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.618634 25028 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:42023
I20260812 05:15:58.618696 25164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:42023 every 8 connection(s)
I20260812 05:15:58.619073 25028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:15:58.622411 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25028
I20260812 05:15:58.622531 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:15:58.622737 19099 external_mini_cluster.cc:1428] Setting key 91b62e9f6a99b1901f85dd4c80bddcd7
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:58.631197 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.619958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:59073 (local address 127.18.166.254:41095)
0812 05:15:58.620061 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:58.620064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:58.620266 (+ 202us) server_negotiation.cc:407] Connection header received
0812 05:15:58.621326 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:58.621327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:58.621341 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:58.621435 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:58.622171 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.622981 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.624316 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.624665 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.624748 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.627601 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:58.627604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:58.627604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:58.627615 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:58.629516 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.630054 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.630055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.630056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.630143 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.630412 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.630413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.630414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.630584 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:58.630666 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:58.630944 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 05:15:58.631058 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:15:58.631229 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.619902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41095 (local address 127.18.166.193:59073)
0812 05:15:58.620127 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:58.620170 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:58.621184 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:58.621526 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:58.621530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:58.621845 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:58.622047 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:58.622051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.623114 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.623115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:58.624200 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:58.624204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.624768 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.624769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:58.624889 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.625566 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:58.625575 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:58.627472 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:58.629658 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:58.629670 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:58.629673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:58.629963 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:58.630224 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:58.630225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:58.630225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:58.630344 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:58.630658 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:58.630664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:58.630801 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 05:15:58.630891 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:15:58.631034 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260812 05:15:58.631930 25165 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41095
I20260812 05:15:58.632063 25165 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:58.632324 25165 heartbeater.cc:511] Master 127.18.166.254:41095 requested a full tablet report, sending...
I20260812 05:15:58.632985 24964 ts_manager.cc:194] Registered new tserver with Master: 4f73eb0be46a4a159ed01bf75401e4c4 (127.18.166.193:42023)
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:15:58.633806 24964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:59073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:15:58.653208 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41095
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:58.738996 25170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:58.739233 25170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:58.739260 25170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:58.740537 25170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:58.740592 25170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:58.740622 25170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:15:58.742274 25170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:58.742540 25170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:58.742811 25170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:58.748241 25175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.748374 25178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.748242 25176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:58.748476 25170 server_base.cc:1034] running on GCE node
I20260812 05:15:58.748842 25170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:58.749166 25170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:58.750375 25170 hybrid_clock.cc:648] HybridClock initialized: now 1786511758750335 us; error 77 us; skew 500 ppm
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:58.753304 25170 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:15:58.753926 25170 webserver.cc:460] Webserver started at http://127.18.166.194:34177/ using document root <none> and password file <none>
I20260812 05:15:58.754194 25170 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:58.754271 25170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:58.754431 25170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:58.755581 25170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "8899e5624fcc4f88b5dd27c182464551"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "6149d730d2a047ed7727221d4a7865dc"
server_key_iv: "2a99b4dcd0eaaaefff7db6975b8b9b26"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.755970 25170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "8899e5624fcc4f88b5dd27c182464551"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "6149d730d2a047ed7727221d4a7865dc"
server_key_iv: "2a99b4dcd0eaaaefff7db6975b8b9b26"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.757556 25170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:15:58.758472 25185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.758646 25170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:58.758754 25170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "8899e5624fcc4f88b5dd27c182464551"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "6149d730d2a047ed7727221d4a7865dc"
server_key_iv: "2a99b4dcd0eaaaefff7db6975b8b9b26"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.758841 25170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:58.774199 25170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:58.776450 25170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:58.776657 25170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:58.776925 25170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:58.777278 25170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:58.777335 25170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.777374 25170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:58.777462 25170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.783876 25170 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41229
I20260812 05:15:58.783927 25302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41229 every 8 connection(s)
I20260812 05:15:58.784320 25170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:15:58.789198 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25170
I20260812 05:15:58.789330 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:15:58.789602 19099 external_mini_cluster.cc:1428] Setting key 4b63fd1af88a6dc75d0d083760524ff6
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:58.796499 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.785205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:34801 (local address 127.18.166.254:41095)
0812 05:15:58.785318 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:58.785320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:58.785486 (+ 166us) server_negotiation.cc:407] Connection header received
0812 05:15:58.786574 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:58.786576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:58.786589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:58.786660 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:58.787443 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.787984 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.789167 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.789536 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.789644 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.792508 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:58.792510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:58.792510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:58.792521 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:58.794457 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.795165 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.795168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.795168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.795243 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.795637 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.795638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.795639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.795821 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:58.795910 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:58.796238 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 05:15:58.796346 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 05:15:58.796638 25305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.785131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41095 (local address 127.18.166.194:34801)
0812 05:15:58.785314 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:58.785358 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:58.786423 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:58.786755 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:58.786760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:58.787067 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:58.787278 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:58.787282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.788115 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.788117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:58.789030 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:58.789033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.789699 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.789700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:58.789847 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.790553 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:58.790563 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:58.792365 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:58.794652 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:58.794668 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:58.794672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:58.795049 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:58.795376 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:58.795377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:58.795377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:58.795541 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:58.795906 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:58.795914 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:58.796063 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 05:15:58.796172 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 05:15:58.796333 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260812 05:15:58.797317 25303 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41095
I20260812 05:15:58.797595 25303 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:58.797818 25303 heartbeater.cc:511] Master 127.18.166.254:41095 requested a full tablet report, sending...
I20260812 05:15:58.798296 24964 ts_manager.cc:194] Registered new tserver with Master: 8899e5624fcc4f88b5dd27c182464551 (127.18.166.194:41229)
I20260812 05:15:58.798866 24964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:34801
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:15:58.821094 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41095
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:15:58.907871 25308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:15:58.908138 25308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:15:58.908169 25308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:15:58.909639 25308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:15:58.909708 25308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:15:58.909744 25308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:15:58.911546 25308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:15:58.911883 25308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:15:58.912163 25308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:15:58.917889 25313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.917901 25316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:15:58.917907 25314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:15:58.918218 25308 server_base.cc:1034] running on GCE node
I20260812 05:15:58.918478 25308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:15:58.918778 25308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:15:58.919948 25308 hybrid_clock.cc:648] HybridClock initialized: now 1786511758919933 us; error 30 us; skew 500 ppm
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:58.922947 25308 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:15:58.923523 25308 webserver.cc:460] Webserver started at http://127.18.166.195:39729/ using document root <none> and password file <none>
I20260812 05:15:58.923858 25308 fs_manager.cc:359] Metadata directory not provided
I20260812 05:15:58.923928 25308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:15:58.924131 25308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:15:58.925331 25308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "71225bd2a4674261a39509fa40bd09be"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "e51e6a79e6a272252095e70359201826"
server_key_iv: "554ae2f3d5af35d5bc695140f8297fb8"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.925839 25308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "71225bd2a4674261a39509fa40bd09be"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "e51e6a79e6a272252095e70359201826"
server_key_iv: "554ae2f3d5af35d5bc695140f8297fb8"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.927469 25308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:15:58.928481 25323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.928808 25308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:15:58.928951 25308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "71225bd2a4674261a39509fa40bd09be"
format_stamp: "Formatted at 2026-08-12 05:15:58 on dist-test-slave-srgv"
server_key: "e51e6a79e6a272252095e70359201826"
server_key_iv: "554ae2f3d5af35d5bc695140f8297fb8"
server_key_version: "encryptionkey@0"
I20260812 05:15:58.929042 25308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:15:58.946810 25308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:15:58.949156 25308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:15:58.949337 25308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:15:58.949756 25308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:15:58.950166 25308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:15:58.950206 25308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.950254 25308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:15:58.950282 25308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:15:58.956907 25308 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:37561
I20260812 05:15:58.957020 25439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:37561 every 8 connection(s)
I20260812 05:15:58.957377 25308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:15:58.967236 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25308
I20260812 05:15:58.967363 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:15:58.967571 19099 external_mini_cluster.cc:1428] Setting key cf344053cc88580f0abfcd29730a320c
I20260812 05:15:58.967693 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.958320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:51153 (local address 127.18.166.254:41095)
0812 05:15:58.958410 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:58.958413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:58.958588 (+ 175us) server_negotiation.cc:407] Connection header received
0812 05:15:58.959572 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:58.959573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:58.959585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:58.959644 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:58.960298 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.960884 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.962059 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.962256 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.962316 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.964799 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:58.964801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:58.964801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:58.964810 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:58.966177 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.966618 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.966619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.966619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.966708 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:58.966944 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:58.966944 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:58.966945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:58.967062 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:58.967117 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:58.967496 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 05:15:58.967574 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 05:15:58.967988 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:58.958272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41095 (local address 127.18.166.195:51153)
0812 05:15:58.958456 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:15:58.958497 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:15:58.959464 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:15:58.959705 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:15:58.959708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:15:58.959998 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:15:58.960211 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:58.960215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.960984 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.960985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:58.961972 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:15:58.961976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:15:58.962342 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:15:58.962342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:15:58.962449 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:58.963103 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:15:58.963115 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:15:58.964688 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:15:58.966266 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:58.966275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:58.966277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:58.966535 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:58.966772 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:15:58.966772 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:15:58.966773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:15:58.966885 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:15:58.967130 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:15:58.967138 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:15:58.967345 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 05:15:58.967482 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 05:15:58.967699 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":115,"threads_started":1}
I20260812 05:15:58.968672 25440 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41095
I20260812 05:15:58.968793 25440 heartbeater.cc:464] Registering TS with master...
I20260812 05:15:58.969000 25440 heartbeater.cc:511] Master 127.18.166.254:41095 requested a full tablet report, sending...
I20260812 05:15:58.969470 24964 ts_manager.cc:194] Registered new tserver with Master: 71225bd2a4674261a39509fa40bd09be (127.18.166.195:37561)
I20260812 05:15:58.969897 24964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:51153
I20260812 05:15:58.979637 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/test-admin.keytab.
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:58.998308 19099 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 05:15:59.000250 25448 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:58.999290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.18.166.193:42023)
0812 05:15:58.999526 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:58.999531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:58.999543 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:15:58.999583 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:58.999585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:58.999600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:58.999676 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:59.000142 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40544: BlockingRecv error: recv got EOF from 127.0.0.1:40544 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":123,"threads_started":1}
W20260812 05:15:59.000413 19099 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc' of type FILE
Aug 12 05:15:58 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:59.001752 19099 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:59.002875 19099 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:59.012481 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:59.003559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.18.166.193:42023)
0812 05:15:59.003624 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:59.003625 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:59.003644 (+ 19us) server_negotiation.cc:407] Connection header received
0812 05:15:59.003720 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:59.003721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:59.003732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:59.003792 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:59.004371 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:59.005349 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:59.006214 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:59.006604 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:59.006669 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:59.009257 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:59.009271 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:59.009271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:59.009280 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:59.011289 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:59.011702 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:59.011705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:59.011705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:59.011763 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:59.011951 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:59.011951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:59.011951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:59.012129 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:59.012237 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:59.012326 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 05:15:59.012398 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 05:15:59.014681 25448 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:59.013976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.18.166.193:42023)
0812 05:15:59.014016 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:59.014018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:59.014105 (+ 87us) server_negotiation.cc:407] Connection header received
0812 05:15:59.014180 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:59.014181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:59.014199 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:59.014265 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:59.014624 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40550: BlockingRecv error: recv got EOF from 127.0.0.1:40550 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 05:15:59.014829 19099 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc' of type FILE
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:59.015974 19099 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:59.017004 19099 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:59.025159 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:59.017690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.18.166.193:42023)
0812 05:15:59.017752 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:59.017753 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:59.017802 (+ 49us) server_negotiation.cc:407] Connection header received
0812 05:15:59.017827 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:59.017828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:59.017838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:59.017882 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:59.018363 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:59.018907 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:59.019651 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:59.019878 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:59.019949 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:59.022347 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:59.022348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:59.022349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:59.022361 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:59.023920 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:59.024300 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:59.024301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:59.024302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:59.024358 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:59.024708 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:59.024709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:59.024709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:59.024826 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:59.024914 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:59.025018 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 05:15:59.025088 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 05:15:59.027069 25448 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:15:59.026439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.18.166.193:42023)
0812 05:15:59.026474 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:59.026476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:59.026484 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:15:59.026535 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:59.026536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:59.026544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:59.026583 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:59.027014 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40556: BlockingRecv error: recv got EOF from 127.0.0.1:40556 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 05:15:59.027166 19099 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786511731160932-19099-0/krb5kdc/krb5cc' of type FILE
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:59.028286 19099 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:15:59.029269 19099 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[24918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:15:59.037313 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:15:59.029911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.18.166.193:42023)
0812 05:15:59.029975 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:15:59.029976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:15:59.030047 (+ 71us) server_negotiation.cc:407] Connection header received
0812 05:15:59.030090 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:15:59.030091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:15:59.030100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:15:59.030147 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:15:59.030572 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:59.031064 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:59.031771 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:15:59.032005 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:15:59.032100 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:15:59.034526 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:15:59.034528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:15:59.034528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:15:59.034537 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:15:59.036105 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:59.036454 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:59.036455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:59.036455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:59.036514 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:15:59.036754 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:15:59.036754 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:15:59.036754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:15:59.036857 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:15:59.036937 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:15:59.037104 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 05:15:59.037170 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:15:59.038141 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25028
I20260812 05:15:59.045758 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25170
I20260812 05:15:59.052263 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25308
I20260812 05:15:59.058573 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 24931
2026-08-12T05:15:59Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2018 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[25461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[25461](info): set up 2 sockets
Aug 12 05:15:59 dist-test-slave-srgv krb5kdc[25461](info): commencing operation
krb5kdc: starting...
W20260812 05:16:00.140580 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:16:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab.
Entry for principal oryx/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab.
I20260812 05:16:00.289469 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35127
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab
--principal=oryx/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:00.377259 25474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:00.377653 25474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:00.377707 25474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:00.379206 25474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:00.379339 25474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:00.379367 25474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:00.379391 25474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:00.379413 25474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:00.381485 25474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab
--principal=oryx/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:00.381892 25474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:00.382191 25474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:00.388065 25479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:00.388278 25474 server_base.cc:1034] running on GCE node
W20260812 05:16:00.388171 25482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.388072 25480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:00.388731 25474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:00.389123 25474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:00.390339 25474 hybrid_clock.cc:648] HybridClock initialized: now 1786511760390270 us; error 86 us; skew 500 ppm
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:00.393586 25474 init.cc:377] Logged in from keytab as oryx/127.18.166.254@KRBTEST.COM (short username oryx)
I20260812 05:16:00.394191 25474 webserver.cc:460] Webserver started at http://127.18.166.254:33709/ using document root <none> and password file <none>
I20260812 05:16:00.394507 25474 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:00.394598 25474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:00.394804 25474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:00.396055 25474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "4cbc350ebb29477ba381a76b5abd0ed7"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "d80fcccd1fb75868fe41e76489dcd79e"
server_key_iv: "ddf30cc6029c265a85e8fb68b5a2a36b"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.396483 25474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "4cbc350ebb29477ba381a76b5abd0ed7"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "d80fcccd1fb75868fe41e76489dcd79e"
server_key_iv: "ddf30cc6029c265a85e8fb68b5a2a36b"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.398308 25474 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:16:00.399471 25489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.399884 25474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:00.400040 25474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "4cbc350ebb29477ba381a76b5abd0ed7"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "d80fcccd1fb75868fe41e76489dcd79e"
server_key_iv: "ddf30cc6029c265a85e8fb68b5a2a36b"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.400173 25474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:00.420289 25474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:00.422941 25474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:00.423178 25474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:00.428164 25474 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:35127
I20260812 05:16:00.428385 25543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:35127 every 8 connection(s)
I20260812 05:16:00.428689 25474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:00.430012 25544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:00.432706 25544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Bootstrap starting.
I20260812 05:16:00.433385 25544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:00.433781 25544 log.cc:824] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:00.434764 25544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: No bootstrap required, opened a new log
I20260812 05:16:00.435596 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25474
I20260812 05:16:00.435750 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:16:00.435951 19099 external_mini_cluster.cc:1428] Setting key f225e6e7359d7242d46bcd4ea3f6fdb4
I20260812 05:16:00.437150 25544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } }
I20260812 05:16:00.437538 25544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:00.437628 25544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cbc350ebb29477ba381a76b5abd0ed7, State: Initialized, Role: FOLLOWER
I20260812 05:16:00.437804 25544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } }
I20260812 05:16:00.437897 25544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:00.437924 25544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:00.438000 25544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:00.439116 25544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } }
I20260812 05:16:00.439287 25544 leader_election.cc:302] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cbc350ebb29477ba381a76b5abd0ed7; no voters:
I20260812 05:16:00.439539 25544 leader_election.cc:288] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:16:00.439571 25550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:00.439704 25550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 LEADER]: Becoming Leader. State: Replica: 4cbc350ebb29477ba381a76b5abd0ed7, State: Running, Role: LEADER
I20260812 05:16:00.439805 25550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } }
I20260812 05:16:00.439909 25544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:00.440200 25550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cbc350ebb29477ba381a76b5abd0ed7. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } }
I20260812 05:16:00.440295 25550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:00.440440 25550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } }
I20260812 05:16:00.440519 25550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
W20260812 05:16:00.441962 25566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:16:00.442070 25566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:16:00.442121 25555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:00.442312 25555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:00.444416 25555 catalog_manager.cc:1347] Generated new cluster ID: 116db59afc0442879118835f9a646747
I20260812 05:16:00.444507 25555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:00.449761 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.436646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.18.166.254:35127)
0812 05:16:00.436845 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:00.436855 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:00.436888 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:16:00.437079 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:00.437086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:00.437141 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:00.437345 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:00.438301 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.439331 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.440441 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.441118 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.441244 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.444790 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:00.444794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:00.444795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:00.444809 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:00.447152 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.447712 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.447715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.447716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.447872 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.448118 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.448119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.448119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.448477 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:00.449224 (+ 747us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:00.449308 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 05:16:00.449517 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260812 05:16:00.450944 25555 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:16:00.451507 25555 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:00.454339 25555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Generated new TSK 0
I20260812 05:16:00.454595 25555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab.
Entry for principal oryx/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab.
I20260812 05:16:00.485005 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35127
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:00.577623 25571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:00.577883 25571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:00.577914 25571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:00.579497 25571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:00.579638 25571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:00.579716 25571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:00.581843 25571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:00.582216 25571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:00.582494 25571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:00.588485 25577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.588564 25579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.588447 25576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:00.588861 25571 server_base.cc:1034] running on GCE node
I20260812 05:16:00.589138 25571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:00.589517 25571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:00.590696 25571 hybrid_clock.cc:648] HybridClock initialized: now 1786511760590670 us; error 47 us; skew 500 ppm
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:00.594003 25571 init.cc:377] Logged in from keytab as oryx/127.18.166.193@KRBTEST.COM (short username oryx)
I20260812 05:16:00.594610 25571 webserver.cc:460] Webserver started at http://127.18.166.193:46125/ using document root <none> and password file <none>
I20260812 05:16:00.594889 25571 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:00.594946 25571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:00.595155 25571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:00.596383 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "5989a69309634f8b910185d08e18992d"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "c7ccddfb4a9b437c76ef0fbb08d14043"
server_key_iv: "3934db76cc49006fc9025ece6cb1ceef"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.596798 25571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "5989a69309634f8b910185d08e18992d"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "c7ccddfb4a9b437c76ef0fbb08d14043"
server_key_iv: "3934db76cc49006fc9025ece6cb1ceef"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.598547 25571 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:16:00.599509 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.599774 25571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:00.599884 25571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "5989a69309634f8b910185d08e18992d"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "c7ccddfb4a9b437c76ef0fbb08d14043"
server_key_iv: "3934db76cc49006fc9025ece6cb1ceef"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.599975 25571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:00.615128 25571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:00.618043 25571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:00.618258 25571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:00.618559 25571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:00.618911 25571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:00.618947 25571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.618973 25571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:00.619022 25571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.626407 25571 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:36853
I20260812 05:16:00.626477 25705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:36853 every 8 connection(s)
I20260812 05:16:00.626911 25571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:00.631928 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25571
I20260812 05:16:00.632113 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:16:00.632351 19099 external_mini_cluster.cc:1428] Setting key ede6f7d160b169565cc5259122fb6a69
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:00.640199 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.628002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:55945 (local address 127.18.166.254:35127)
0812 05:16:00.628060 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:00.628063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:00.628356 (+ 293us) server_negotiation.cc:407] Connection header received
0812 05:16:00.629036 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:00.629038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:00.629057 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:00.629143 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:00.630542 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.631153 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.632794 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.633090 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.633205 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.636295 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:00.636298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:00.636298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:00.636313 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:00.638324 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.638954 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.638956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.638956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.639035 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.639372 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.639373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.639373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.639524 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:00.639615 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:00.639959 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 05:16:00.640081 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:16:00.640460 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.627951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:55945)
0812 05:16:00.628169 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:00.628235 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:00.628823 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:00.629267 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:00.629273 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:00.630085 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:00.630352 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:00.630359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.631330 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.631331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:00.632630 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:00.632634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.633261 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.633263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:00.633510 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.634219 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:00.634231 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:00.636125 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:00.638453 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:00.638473 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:00.638477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:00.638805 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:00.639129 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:00.639130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:00.639131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:00.639290 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:00.639615 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:00.639628 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:00.639795 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 05:16:00.639965 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 05:16:00.640178 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":121,"threads_started":1}
I20260812 05:16:00.641227 25706 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35127
I20260812 05:16:00.641341 25706 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:00.641629 25706 heartbeater.cc:511] Master 127.18.166.254:35127 requested a full tablet report, sending...
I20260812 05:16:00.642673 25508 ts_manager.cc:194] Registered new tserver with Master: 5989a69309634f8b910185d08e18992d (127.18.166.193:36853)
WARNING: no policy specified for oryx/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:16:00.643747 25508 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:55945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab.
Entry for principal oryx/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab.
I20260812 05:16:00.664850 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35127
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:00.752447 25711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:00.752713 25711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:00.752744 25711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:00.754220 25711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:00.754297 25711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:00.754329 25711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:00.756188 25711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:00.756526 25711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:00.756875 25711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:00.762476 25719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.762493 25716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.762476 25717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:00.762737 25711 server_base.cc:1034] running on GCE node
I20260812 05:16:00.762971 25711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:00.763330 25711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:00.764534 25711 hybrid_clock.cc:648] HybridClock initialized: now 1786511760764519 us; error 43 us; skew 500 ppm
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:00.767781 25711 init.cc:377] Logged in from keytab as oryx/127.18.166.194@KRBTEST.COM (short username oryx)
I20260812 05:16:00.768421 25711 webserver.cc:460] Webserver started at http://127.18.166.194:37999/ using document root <none> and password file <none>
I20260812 05:16:00.768729 25711 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:00.768811 25711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:00.768989 25711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:00.770318 25711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "00365876de384b3b8745214b55861708"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "cb4a0ba17e49290e7ed9684951eaa248"
server_key_iv: "8644ffebd37afe5283dcf75554d89aea"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.770750 25711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "00365876de384b3b8745214b55861708"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "cb4a0ba17e49290e7ed9684951eaa248"
server_key_iv: "8644ffebd37afe5283dcf75554d89aea"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.772684 25711 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 05:16:00.773911 25726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.774158 25711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:00.774305 25711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "00365876de384b3b8745214b55861708"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "cb4a0ba17e49290e7ed9684951eaa248"
server_key_iv: "8644ffebd37afe5283dcf75554d89aea"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.774426 25711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:00.786935 25711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:00.791342 25711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:00.791548 25711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:00.791836 25711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:00.792225 25711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:00.792286 25711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.792323 25711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:00.792352 25711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.799305 25711 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38983
I20260812 05:16:00.799353 25846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38983 every 8 connection(s)
I20260812 05:16:00.799801 25711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:00.801610 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25711
I20260812 05:16:00.801726 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:16:00.801955 19099 external_mini_cluster.cc:1428] Setting key e160218b5463032454f342637bc08862
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:16:00.812480 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.800738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:41065 (local address 127.18.166.254:35127)
0812 05:16:00.800844 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:00.800846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:00.801041 (+ 195us) server_negotiation.cc:407] Connection header received
0812 05:16:00.801634 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:00.801636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:00.801652 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:00.801738 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:00.803126 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.803916 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.805126 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.805540 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.805628 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.808744 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:00.808746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:00.808746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:00.808757 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:00.810649 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.811238 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.811239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.811240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.811311 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.811643 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.811644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.811644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.811787 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:00.811873 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:00.812247 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 05:16:00.812368 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 05:16:00.812716 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.800692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:41065)
0812 05:16:00.800906 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:00.800950 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:00.801524 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:00.801845 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:00.801849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:00.802689 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:00.802992 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:00.802998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.804042 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.804043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:00.805005 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:00.805008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.805694 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.805695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:00.805888 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.806605 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:00.806617 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:00.808602 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:00.810803 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:00.810818 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:00.810822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:00.811126 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:00.811400 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:00.811401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:00.811402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:00.811558 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:00.811860 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:00.811869 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:00.812093 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 05:16:00.812227 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 05:16:00.812453 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
I20260812 05:16:00.813787 25847 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35127
I20260812 05:16:00.813895 25847 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:00.814074 25847 heartbeater.cc:511] Master 127.18.166.254:35127 requested a full tablet report, sending...
I20260812 05:16:00.814452 25508 ts_manager.cc:194] Registered new tserver with Master: 00365876de384b3b8745214b55861708 (127.18.166.194:38983)
I20260812 05:16:00.814953 25508 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:41065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab.
Entry for principal oryx/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab.
I20260812 05:16:00.833876 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35127
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:00.923209 25852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:00.923484 25852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:00.923514 25852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:00.925035 25852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:00.925117 25852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:00.925151 25852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:00.927019 25852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:00.927389 25852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:00.927678 25852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:00.933298 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.933303 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:00.933303 25857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:00.933653 25852 server_base.cc:1034] running on GCE node
I20260812 05:16:00.933966 25852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:00.934264 25852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:00.935436 25852 hybrid_clock.cc:648] HybridClock initialized: now 1786511760935416 us; error 56 us; skew 500 ppm
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:00.938580 25852 init.cc:377] Logged in from keytab as oryx/127.18.166.195@KRBTEST.COM (short username oryx)
I20260812 05:16:00.939126 25852 webserver.cc:460] Webserver started at http://127.18.166.195:36579/ using document root <none> and password file <none>
I20260812 05:16:00.939422 25852 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:00.939504 25852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:00.939687 25852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:00.941123 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "d72cb9c007d54c6ca0150247f1251ea0"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "12ad673c9c4913cb869c88ff9e202020"
server_key_iv: "8d61498da0b434a3b259f409c6ffd899"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.941737 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "d72cb9c007d54c6ca0150247f1251ea0"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "12ad673c9c4913cb869c88ff9e202020"
server_key_iv: "8d61498da0b434a3b259f409c6ffd899"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.943432 25852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:00.944509 25868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.944772 25852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:00.944918 25852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "d72cb9c007d54c6ca0150247f1251ea0"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "12ad673c9c4913cb869c88ff9e202020"
server_key_iv: "8d61498da0b434a3b259f409c6ffd899"
server_key_version: "encryptionkey@0"
I20260812 05:16:00.945017 25852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:00.967649 25852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:00.970393 25852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:00.970597 25852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:00.970866 25852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:00.971215 25852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:00.971280 25852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.971318 25852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:00.971362 25852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:00.977783 25852 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:46791
I20260812 05:16:00.977828 25987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:46791 every 8 connection(s)
I20260812 05:16:00.978229 25852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:00.980417 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 25852
I20260812 05:16:00.980540 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:16:00.980726 19099 external_mini_cluster.cc:1428] Setting key 38874d16b66339e1acb6a2d5b40a0a0a
Aug 12 05:16:00 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:00.988734 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.979136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:53747 (local address 127.18.166.254:35127)
0812 05:16:00.979185 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:00.979187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:00.979452 (+ 265us) server_negotiation.cc:407] Connection header received
0812 05:16:00.980031 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:00.980032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:00.980046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:00.980103 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:00.981377 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.981934 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.983009 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.983214 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.983306 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.985863 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:00.985865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:00.985865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:00.985873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:00.987168 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.987651 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.987653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.987653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.987721 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:00.988003 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:00.988004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:00.988004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:00.988136 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:00.988218 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:00.988538 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 05:16:00.988635 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:16:00.988991 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:00.979132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:53747)
0812 05:16:00.979317 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:00.979355 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:00.979909 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:00.980181 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:00.980204 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:00.980948 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:00.981284 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:00.981291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.982055 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.982056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:00.982895 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:00.982898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:00.983317 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:00.983317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:00.983459 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:00.984019 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:00.984027 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:00.985719 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:00.987278 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:00.987288 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:00.987291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:00.987558 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:00.987794 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:00.987795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:00.987796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:00.987921 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:00.988220 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:00.988228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:00.988386 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 05:16:00.988514 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 05:16:00.988694 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260812 05:16:00.989652 25989 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35127
I20260812 05:16:00.989779 25989 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:00.990005 25989 heartbeater.cc:511] Master 127.18.166.254:35127 requested a full tablet report, sending...
I20260812 05:16:00.990444 25508 ts_manager.cc:194] Registered new tserver with Master: d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:00.990811 25508 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:53747
I20260812 05:16:00.992538 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.166.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.253.keytab.
Entry for principal oryx/127.18.166.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.253.keytab.
I20260812 05:16:01.022524 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:37805
--webserver_interface=127.18.166.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.253.keytab
--principal=oryx/127.18.166.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35127,127.18.166.253:37805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:01.111801 26001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:01.112059 26001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:01.112087 26001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:01.113394 26001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:01.113544 26001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:01.113570 26001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:01.113595 26001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:01.113617 26001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:01.115464 26001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35127,127.18.166.253:37805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.253.keytab
--principal=oryx/127.18.166.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:37805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.166.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
W20260812 05:16:01.115602 26001 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.166.254:35127,127.18.166.253:37805'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 05:16:01.115734 26001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:01.115943 26001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:16:01.121392 26001 server_base.cc:1034] running on GCE node
W20260812 05:16:01.121371 26007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.121367 26009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.121349 26006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.121944 26001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:01.122269 26001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:01.123461 26001 hybrid_clock.cc:648] HybridClock initialized: now 1786511761123433 us; error 43 us; skew 500 ppm
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:01.126600 26001 init.cc:377] Logged in from keytab as oryx/127.18.166.253@KRBTEST.COM (short username oryx)
I20260812 05:16:01.127189 26001 webserver.cc:460] Webserver started at http://127.18.166.253:40793/ using document root <none> and password file <none>
I20260812 05:16:01.127440 26001 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.127490 26001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.127599 26001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:01.128765 26001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/instance:
uuid: "25e23785489c43d0a39e77a995650c42"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "2ac4328cc6b31cdc017508c122237fb0"
server_key_iv: "4667d297efcf6781b433d713e3bba191"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.129123 26001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal/instance:
uuid: "25e23785489c43d0a39e77a995650c42"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "2ac4328cc6b31cdc017508c122237fb0"
server_key_iv: "4667d297efcf6781b433d713e3bba191"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.130784 26001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:16:01.131867 26016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.132083 26001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.132170 26001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
uuid: "25e23785489c43d0a39e77a995650c42"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "2ac4328cc6b31cdc017508c122237fb0"
server_key_iv: "4667d297efcf6781b433d713e3bba191"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.132287 26001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.140388 26001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:01.143214 26001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:01.143404 26001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:01.148495 26001 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.253:37805
I20260812 05:16:01.148523 26071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.253:37805 every 8 connection(s)
I20260812 05:16:01.148985 26001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
I20260812 05:16:01.150306 26072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:01.152037 26072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.253@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:01.158854 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 26001
I20260812 05:16:01.162762 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.152894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.18.166.254:35127)
0812 05:16:01.152965 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.152968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.153220 (+ 252us) server_negotiation.cc:407] Connection header received
0812 05:16:01.153822 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.153824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.153837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.153899 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.154641 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.155163 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.156281 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.156502 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.156567 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.159235 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.159237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.159238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.159248 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.160951 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.161596 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.161597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.161598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.161686 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.161972 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.161973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.161974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.162152 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.162236 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.162547 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.162653 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:16:01.163008 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.152868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45026)
0812 05:16:01.153072 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.153114 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.153699 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.153985 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.153990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.154288 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.154508 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.154512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.155288 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.155289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.156169 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.156172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.156602 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.156603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.156727 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.157279 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.157288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.159112 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.161146 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.161155 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.161157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.161489 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.161759 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.161760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.161761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.161896 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.162239 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.162245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.162401 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.162511 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.162747 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":27,"thread_start_us":116,"threads_started":1}
I20260812 05:16:01.163956 26072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.253@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
I20260812 05:16:01.169396 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.159450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44632 (local address 127.18.166.253:37805)
0812 05:16:01.159961 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.159966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.159987 (+ 21us) server_negotiation.cc:407] Connection header received
0812 05:16:01.160010 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.160012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.160024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.160123 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.160856 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.161667 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.162449 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.162797 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.162890 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.165945 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.165956 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.165956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.165964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.167943 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.168400 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.168404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.168405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.168481 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.168727 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.168728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.168729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.168913 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.169117 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.169166 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.169260 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":478,"thread_start_us":110,"threads_started":1}
I20260812 05:16:01.172566 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.164339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.18.166.253:37805)
0812 05:16:01.164524 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.164526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.164531 (+ 5us) server_negotiation.cc:407] Connection header received
0812 05:16:01.164545 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.164545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.164552 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.164602 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.164937 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.165687 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.166448 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.166670 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.166750 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.168889 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.168890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.168890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.168897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.171268 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.171655 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.171656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.171657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.171723 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.171929 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.171930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.171930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.172052 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.172147 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.172291 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.172401 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":161,"thread_start_us":136,"threads_started":1}
I20260812 05:16:01.172566 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.164304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44646)
0812 05:16:01.164337 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.164348 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.164392 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.164644 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.164646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.164794 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.164847 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.164849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.165773 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.165775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.166343 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.166345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.166745 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.166746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.166808 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.167343 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.167348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.168772 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.171382 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.171383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.171383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.171561 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.171799 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.171800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.171800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.171859 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.172139 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.172140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.172180 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.172268 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.172467 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":101}
I20260812 05:16:01.174185 26072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Bootstrap starting.
I20260812 05:16:01.174911 26072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:01.175221 26072 log.cc:824] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:01.176255 26072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: No bootstrap required, opened a new log
I20260812 05:16:01.178499 26072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } }
I20260812 05:16:01.178769 26072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:01.178830 26072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e23785489c43d0a39e77a995650c42, State: Initialized, Role: FOLLOWER
I20260812 05:16:01.178983 26072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } }
I20260812 05:16:01.179354 26079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } } }
I20260812 05:16:01.179419 26072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:01.179450 26079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:16:01.181195 26091 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:16:01.181278 26091 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:16:01.183264 26001 master_runner.cc:428] Detected that this master 25e23785489c43d0a39e77a995650c42 is joining an existing cluster
I20260812 05:16:01.183377 26001 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.166.253:37805
I20260812 05:16:01.192350 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.184765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.18.166.254:35127)
0812 05:16:01.184812 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.184815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.185138 (+ 323us) server_negotiation.cc:407] Connection header received
0812 05:16:01.185204 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.185205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.185219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.185290 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.185942 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.186792 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.187917 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.188211 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.188297 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.189378 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.189380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.189380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.189391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.191024 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191434 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.191435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.191435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.191491 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191762 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.191763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.191764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.191900 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.191965 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.192128 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.192249 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:16:01.192371 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.184709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44652)
0812 05:16:01.185029 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.185051 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.185135 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.185324 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.185325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.185625 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.185722 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.185725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.186692 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.186692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.187324 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.187327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.187762 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.187763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.187822 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.188431 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.188435 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.188934 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.190972 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.190974 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.190975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.191212 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.191560 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.191561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.191619 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.191908 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.191909 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.191964 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.192038 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.192186 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":105,"threads_started":1}
I20260812 05:16:01.192745 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.184859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45028)
0812 05:16:01.185029 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.185044 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.185133 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.185395 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.185395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.185655 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.185750 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.185751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.186897 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.186899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.187810 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.187814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.188311 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.188313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.188387 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.188843 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.188848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.189269 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.191115 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.191116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.191344 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.191565 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.191572 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.191657 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.191955 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.191955 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.191996 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.192520 (+ 524us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.192650 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
I20260812 05:16:01.192384 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.184763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.18.166.253:37805)
0812 05:16:01.184812 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.184815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.185113 (+ 298us) server_negotiation.cc:407] Connection header received
0812 05:16:01.185203 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.185204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.185214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.185246 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.185821 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.186579 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.187420 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.187670 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.187767 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.189043 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.189044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.189045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.189052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.190864 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191286 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.191287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.191288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.191345 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.191683 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.191684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.191684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.191830 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.191903 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.192076 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.192275 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":23}
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.254@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
I20260812 05:16:01.213510 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.203165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.18.166.253:37805)
0812 05:16:01.203207 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.203209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.203220 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:01.203307 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.203308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.203319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.203371 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.203995 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.204554 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.205471 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.205785 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.205857 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.209469 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.209472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.209472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.209483 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.211594 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.212264 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.212266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.212266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.212345 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.212620 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.212621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.212622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.212803 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.212862 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.213235 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.213352 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:16:01.213981 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.202873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44660)
0812 05:16:01.203054 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.203124 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.203215 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.203378 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.203381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.203791 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.203853 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.203857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.204668 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.204670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.205312 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.205315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.205934 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.205935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.206073 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.206999 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.207006 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.209260 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.211743 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.211759 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.211762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.212152 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.212455 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.212456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.212456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.212532 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.212929 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.212945 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.213030 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.213681 (+ 651us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.213871 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260812 05:16:01.214893 25508 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.166.253:37805: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "25e23785489c43d0a39e77a995650c42"
member_type: NON_VOTER
last_known_addr {
host: "127.18.166.253"
port: 37805
}
attrs {
promote: true
}
}
dest_uuid: "4cbc350ebb29477ba381a76b5abd0ed7"
cas_config_opid_index: -1
I20260812 05:16:01.215399 25508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } }
I20260812 05:16:01.215986 26103 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 25e23785489c43d0a39e77a995650c42 (127.18.166.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } }
I20260812 05:16:01.216778 26045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:01.217875 26045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Refusing update from remote peer 4cbc350ebb29477ba381a76b5abd0ed7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 05:16:01.218303 26101 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:01.218575 26103 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.166.253:37805
I20260812 05:16:01.218791 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } }
I20260812 05:16:01.218864 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:01.218966 26001 master.cc:537] Master@127.18.166.253:37805 shutting down...
W20260812 05:16:01.220914 25490 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 05:16:01.221771 25490 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 -> Peer 25e23785489c43d0a39e77a995650c42 (127.18.166.253:37805): Couldn't send request to peer 25e23785489c43d0a39e77a995650c42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 05:16:01.222368 26001 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:01.222478 26001 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:01.222533 26001 tablet_replica.cc:330] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: stopping tablet replica
I20260812 05:16:01.235869 26001 master.cc:552] Master@127.18.166.253:37805 shutdown complete.
I20260812 05:16:01.235968 26001 master_runner.cc:305] Clearing existing system tablet
I20260812 05:16:01.236119 26001 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.236191 26001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.236917 26001 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:01.237548 26109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.237746 26001 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.237818 26001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:01.237851 26001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
uuid: "25e23785489c43d0a39e77a995650c42"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "2ac4328cc6b31cdc017508c122237fb0"
server_key_iv: "4667d297efcf6781b433d713e3bba191"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.238907 26001 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:01.240934 26001 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 05:16:01.241055 26001 log.cc:1196] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 05:16:01.241298 26001 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Deleting consensus metadata
I20260812 05:16:01.241559 26001 master_runner.cc:315] Copying system tablet from 127.18.166.254:35127
I20260812 05:16:01.242257 26001 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: tablet copy: Beginning tablet copy session from remote peer at address 127.18.166.254:35127
I20260812 05:16:01.251362 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.242670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45038)
0812 05:16:01.242920 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.242933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.243020 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.243187 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.243188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.243428 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.243505 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.243508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.244405 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.244406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.245298 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.245302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.245859 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.245861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.245942 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.246534 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.246540 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.247015 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.249027 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.249028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.249029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.249250 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.249686 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.249687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.249688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.249773 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.250106 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.250107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.250557 (+ 450us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.250883 (+ 326us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.251152 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":119,"threads_started":1}
I20260812 05:16:01.251300 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.242775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.18.166.254:35127)
0812 05:16:01.242852 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.242855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.243018 (+ 163us) server_negotiation.cc:407] Connection header received
0812 05:16:01.243092 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.243093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.243104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.243159 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.243646 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.244281 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.245558 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.245764 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.245834 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.247134 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.247136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.247136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.247145 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.248919 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.249343 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.249345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.249345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.249560 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.249848 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.249850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.249850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.250001 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.250512 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.250974 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.251152 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:16:01.252066 25528 tablet_copy_service.cc:140] P 4cbc350ebb29477ba381a76b5abd0ed7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 25e23785489c43d0a39e77a995650c42 ({username='oryx', principal='oryx/127.18.166.253@KRBTEST.COM'} at 127.0.0.1:45038)
I20260812 05:16:01.252300 25528 tablet_copy_service.cc:161] P 4cbc350ebb29477ba381a76b5abd0ed7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 25e23785489c43d0a39e77a995650c42 at {username='oryx', principal='oryx/127.18.166.253@KRBTEST.COM'} at 127.0.0.1:45038: session id = 25e23785489c43d0a39e77a995650c42-00000000000000000000000000000000
I20260812 05:16:01.253479 25528 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 05:16:01.254578 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 05:16:01.255940 26001 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: tablet copy: Starting download of 0 data blocks...
I20260812 05:16:01.256163 26001 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: tablet copy: Starting download of 1 WAL segments...
I20260812 05:16:01.257619 26001 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 05:16:01.258776 25528 tablet_copy_service.cc:342] P 4cbc350ebb29477ba381a76b5abd0ed7: Request end of tablet copy session 25e23785489c43d0a39e77a995650c42-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.166.253@KRBTEST.COM'} at 127.0.0.1:45038
I20260812 05:16:01.258865 25528 tablet_copy_service.cc:433] P 4cbc350ebb29477ba381a76b5abd0ed7: ending tablet copy session 25e23785489c43d0a39e77a995650c42-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 25e23785489c43d0a39e77a995650c42
I20260812 05:16:01.262290 26001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:01.263675 26116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.263675 26117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.264016 26119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.264036 26001 server_base.cc:1034] running on GCE node
I20260812 05:16:01.264493 26001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:01.264699 26001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:01.265861 26001 hybrid_clock.cc:648] HybridClock initialized: now 1786511761265858 us; error 36 us; skew 500 ppm
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:01.267521 26001 init.cc:377] Logged in from keytab as oryx/127.18.166.253@KRBTEST.COM (short username oryx)
I20260812 05:16:01.267858 26001 webserver.cc:460] Webserver started at http://127.18.166.253:42039/ using document root <none> and password file <none>
I20260812 05:16:01.268014 26001 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.268074 26001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.268819 26001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.269570 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.269728 26001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:01.269780 26001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
uuid: "25e23785489c43d0a39e77a995650c42"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "2ac4328cc6b31cdc017508c122237fb0"
server_key_iv: "4667d297efcf6781b433d713e3bba191"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.269846 26001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.288347 26001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:01.290478 26001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:01.293994 26001 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.253:37805
I20260812 05:16:01.294190 26180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.253:37805 every 8 connection(s)
I20260812 05:16:01.294430 26001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
I20260812 05:16:01.295315 26181 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:16:01.295774 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Bootstrap starting.
I20260812 05:16:01.298488 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:16:01.298751 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Bootstrap complete.
I20260812 05:16:01.298995 26181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } }
I20260812 05:16:01.299114 26181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 25e23785489c43d0a39e77a995650c42, State: Initialized, Role: LEARNER
I20260812 05:16:01.299170 26181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } }
I20260812 05:16:01.299386 26181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:01.299435 26184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } }
I20260812 05:16:01.299556 26184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: LEARNER
I20260812 05:16:01.300957 26195 catalog_manager.cc:1259] Loaded cluster ID: 116db59afc0442879118835f9a646747
I20260812 05:16:01.301090 26195 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: loading cluster ID for follower catalog manager: success
I20260812 05:16:01.302043 26195 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: acquiring CA information for follower catalog manager: success
I20260812 05:16:01.302371 26195 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:16:01.332695 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.325085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.18.166.253:37805)
0812 05:16:01.325260 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.325263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.325271 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:01.325307 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.325307 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.325318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.325402 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.326101 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.326792 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.327588 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.327819 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.327909 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.329468 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.329471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.329471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.329487 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.331307 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.331762 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.331764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.331764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.331832 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.332053 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.332054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.332054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.332187 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.332303 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.332438 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.332535 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for oryx/127.18.166.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.252.keytab.
Entry for principal oryx/127.18.166.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.252.keytab.
I20260812 05:16:01.364974 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:37123
--webserver_interface=127.18.166.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.252.keytab
--principal=oryx/127.18.166.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:01.452121 26199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:01.452370 26199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:01.452397 26199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:01.453965 26199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:01.454061 26199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:01.454093 26199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:01.454113 26199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:01.454135 26199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:01.455906 26199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.252.keytab
--principal=oryx/127.18.166.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:37123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.166.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:01.456173 26199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:01.456386 26199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:01.461666 26204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.461829 26199 server_base.cc:1034] running on GCE node
W20260812 05:16:01.461673 26205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.461673 26207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.462277 26199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:01.462571 26199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:01.463768 26199 hybrid_clock.cc:648] HybridClock initialized: now 1786511761463762 us; error 42 us; skew 500 ppm
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:01.466813 26199 init.cc:377] Logged in from keytab as oryx/127.18.166.252@KRBTEST.COM (short username oryx)
I20260812 05:16:01.467319 26199 webserver.cc:460] Webserver started at http://127.18.166.252:43691/ using document root <none> and password file <none>
I20260812 05:16:01.467540 26199 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.467584 26199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.467690 26199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:01.468843 26199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/instance:
uuid: "de028c7533a3448da2355a31573994b0"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "dd278694bbfce5b4746d2b8bdace99b8"
server_key_iv: "87a33351c10cc01cd83b964dccef594a"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.469173 26199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal/instance:
uuid: "de028c7533a3448da2355a31573994b0"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "dd278694bbfce5b4746d2b8bdace99b8"
server_key_iv: "87a33351c10cc01cd83b964dccef594a"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.470963 26199 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 05:16:01.471863 26214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.472056 26199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.472168 26199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
uuid: "de028c7533a3448da2355a31573994b0"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "dd278694bbfce5b4746d2b8bdace99b8"
server_key_iv: "87a33351c10cc01cd83b964dccef594a"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.472237 26199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.482620 26199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:01.484824 26199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:01.484963 26199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:01.489665 26199 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.252:37123
I20260812 05:16:01.489753 26275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.252:37123 every 8 connection(s)
I20260812 05:16:01.490156 26199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
I20260812 05:16:01.491189 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 26199
I20260812 05:16:01.491595 26276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:01.494421 26276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.166.252@KRBTEST.COM
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.252@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:01.503922 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.491988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.18.166.252:37123)
0812 05:16:01.492206 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.492215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.492245 (+ 30us) server_negotiation.cc:407] Connection header received
0812 05:16:01.492851 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.492859 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.492918 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.493052 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.493986 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.495013 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.495908 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.496525 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.496609 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.499615 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.499624 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.499625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.499638 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.501932 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.502388 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.502393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.502393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.502522 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.502784 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.502785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.502785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.503182 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.503377 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.503495 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.503708 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":103,"threads_started":1}
I20260812 05:16:01.505298 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.495364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.18.166.254:35127)
0812 05:16:01.495423 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.495426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.495564 (+ 138us) server_negotiation.cc:407] Connection header received
0812 05:16:01.495642 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.495643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.495656 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.495735 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.496406 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.497116 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.498030 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.498350 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.498423 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.501629 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.501632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.501632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.501643 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.503385 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.504333 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.504334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.504335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.504411 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.504606 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.504607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.504607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.504810 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.504880 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.505089 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.505197 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 05:16:01.505336 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.495220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45050)
0812 05:16:01.495428 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.495488 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.495580 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.495761 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.495765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.496215 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.496286 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.496288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.497215 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.497216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.497933 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.497936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.498489 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.498491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.498569 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.499677 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.499682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.501513 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.503472 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.503482 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.503484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.504258 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.504490 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.504490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.504491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.504557 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.504868 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.504876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.504937 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.505047 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.505196 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260812 05:16:01.505878 26276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.252@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
I20260812 05:16:01.512861 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.506276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.18.166.253:37805)
0812 05:16:01.506339 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.506341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.506349 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:01.506374 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.506375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.506387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.506429 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.506710 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.507370 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.508053 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.508276 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.508332 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.510396 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.510397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.510397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.510405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.511720 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.512096 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.512096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.512097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.512205 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.512409 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.512409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.512410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.512528 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.512584 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.512710 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.512789 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:16:01.512872 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.506179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44696)
0812 05:16:01.506253 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.506265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.506315 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.506445 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.506445 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.506579 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.506641 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.506642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.507452 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.507453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.507974 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.507975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.508346 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.508346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.508402 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.508940 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.508945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.510302 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.511829 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.511831 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.511831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.512021 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.512286 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.512286 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.512287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.512348 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.512591 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.512592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.512636 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.512690 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.512789 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 05:16:01.513512 26276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.252@KRBTEST.COM for oryx/127.18.166.252@KRBTEST.COM
I20260812 05:16:01.520859 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.513837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34518)
0812 05:16:01.513893 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.513904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.513969 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.514095 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.514096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.514241 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.514273 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.514275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.514972 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.514973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.515464 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.515466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.515868 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.515869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.515916 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.516428 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.516433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.518035 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.519691 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.519693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.519693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.519911 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.520168 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.520168 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.520169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.520243 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.520531 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.520532 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.520578 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.520651 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.520757 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:16:01.520916 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.513903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.18.166.252:37123)
0812 05:16:01.513956 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.513958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.513967 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:01.514026 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.514027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.514036 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.514094 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.514354 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.514855 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.515553 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.515777 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.515870 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.518196 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.518198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.518198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.518206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.519562 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.519988 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.519989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.519989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.520044 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.520321 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.520321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.520321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.520440 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.520496 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.520680 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.520808 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":35}
I20260812 05:16:01.522650 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Bootstrap starting.
I20260812 05:16:01.523605 26276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:01.523923 26276 log.cc:824] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:01.524788 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: No bootstrap required, opened a new log
I20260812 05:16:01.526561 26276 raft_consensus.cc:348] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } }
I20260812 05:16:01.526738 26276 raft_consensus.cc:374] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:01.526782 26276 raft_consensus.cc:729] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de028c7533a3448da2355a31573994b0, State: Initialized, Role: FOLLOWER
I20260812 05:16:01.526950 26276 consensus_queue.cc:260] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } }
I20260812 05:16:01.527344 26276 sys_catalog.cc:564] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:01.527344 26281 sys_catalog.cc:455] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } } }
I20260812 05:16:01.527468 26281 sys_catalog.cc:458] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:16:01.529203 26298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:16:01.529290 26298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:16:01.531944 26199 master_runner.cc:428] Detected that this master de028c7533a3448da2355a31573994b0 is joining an existing cluster
I20260812 05:16:01.532006 26199 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.166.252:37123
I20260812 05:16:01.541821 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.533137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44710)
0812 05:16:01.533650 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.533662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.533733 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.533970 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.533970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.534356 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.534427 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.534430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.535383 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.535384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.536455 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.536460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.537023 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.537023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.537085 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.537770 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.537776 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.538182 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.540434 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.540435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.540436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.540658 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.540901 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.540902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.540902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.540989 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.541384 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.541384 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.541492 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.541593 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.541718 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":74,"spinlock_wait_cycles":10496,"thread_start_us":111,"threads_started":1}
I20260812 05:16:01.541855 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.533253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.18.166.253:37805)
0812 05:16:01.533360 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.533363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.533735 (+ 372us) server_negotiation.cc:407] Connection header received
0812 05:16:01.533845 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.533846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.533861 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.533894 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.534533 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.535278 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.536578 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.536906 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.536949 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.538302 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.538303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.538304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.538313 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.540262 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.540748 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.540749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.540750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.540823 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.541140 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.541141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.541141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.541305 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.541383 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.541641 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.541750 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 05:16:01.555828 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25571
I20260812 05:16:01.565229 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:36853
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:35127
--webserver_port=46125
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:16:01.565390 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.533308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.18.166.254:35127)
0812 05:16:01.533401 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.533403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.533758 (+ 355us) server_negotiation.cc:407] Connection header received
0812 05:16:01.533792 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.533793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.533809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.533874 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.534542 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.535326 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.536581 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.536894 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.536939 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.561607 (+ 24668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.561610 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.561611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.561625 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.563469 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564368 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.564370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.564370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.564441 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564653 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.564653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.564654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.564812 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.564867 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.565141 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.565265 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 05:16:01.565510 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.533492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.18.166.252:37123)
0812 05:16:01.533558 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.533560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.533882 (+ 322us) server_negotiation.cc:407] Connection header received
0812 05:16:01.533964 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.533965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.533972 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.533999 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.534521 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.535278 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.536569 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.536857 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.536901 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.561766 (+ 24865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.561767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.561768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.561775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.564175 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564626 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.564627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.564628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.564679 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564875 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.564876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.564876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.565011 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.565060 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.565245 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.565331 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:16:01.565809 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.533142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45052)
0812 05:16:01.533650 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.533662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.533733 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.533935 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.533936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.534304 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.534435 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.534438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.535438 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.535439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.536464 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.536468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.537030 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.537031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.537133 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.560438 (+ 23305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.560447 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.561358 (+ 911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.563595 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.563597 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.563597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.564285 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.564520 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.564521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.564583 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.564932 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.564933 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.564994 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.565537 (+ 543us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.565676 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":86,"thread_start_us":85,"threads_started":1}
I20260812 05:16:01.566828 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.533458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34530)
0812 05:16:01.533830 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.533839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.533901 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.534056 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.534056 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.534318 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.534427 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.534430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.535383 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.535384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.536455 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.536460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.537006 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.537007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.537073 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.560555 (+ 23482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.560562 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.561633 (+ 1071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.564283 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.564284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.564544 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.564753 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.564754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.564754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.564810 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.565103 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.565104 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.565142 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.566508 (+ 1366us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.566659 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":6,"thread_start_us":115,"threads_started":1}
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511760, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.254@KRBTEST.COM for oryx/127.18.166.252@KRBTEST.COM
I20260812 05:16:01.576730 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.569388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.18.166.252:37123)
0812 05:16:01.569458 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.569460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.569515 (+ 55us) server_negotiation.cc:407] Connection header received
0812 05:16:01.569598 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.569599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.569610 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.569662 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.570025 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.570631 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.571460 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.571678 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.571764 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.573912 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.573913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.573914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.573923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.575421 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.575863 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.575864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.575865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.575937 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.576153 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.576154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.576155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.576287 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.576366 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.576543 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.576634 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:16:01.576798 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.569353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34536)
0812 05:16:01.569468 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.569481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.569534 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.569664 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.569665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.569880 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.569940 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.569943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.570780 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.570781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.571366 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.571368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.571762 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.571764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.571839 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.572312 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.572318 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.573797 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.575569 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.575571 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.575571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.575783 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.576013 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.576014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.576014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.576082 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.576370 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.576371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.576423 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.576504 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.576677 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 05:16:01.577841 25508 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.166.252:37123: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "de028c7533a3448da2355a31573994b0"
member_type: NON_VOTER
last_known_addr {
host: "127.18.166.252"
port: 37123
}
attrs {
promote: true
}
}
dest_uuid: "4cbc350ebb29477ba381a76b5abd0ed7"
cas_config_opid_index: 5
I20260812 05:16:01.578145 25508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } }
I20260812 05:16:01.578817 26101 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER de028c7533a3448da2355a31573994b0 (127.18.166.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.579797 26101 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.166.252:37123
I20260812 05:16:01.579908 26101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.580004 26101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:01.580427 26199 master.cc:537] Master@127.18.166.252:37123 shutting down...
I20260812 05:16:01.580595 26244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:01.582008 26244 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Refusing update from remote peer 4cbc350ebb29477ba381a76b5abd0ed7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 05:16:01.582418 26104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 05:16:01.583237 25491 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 -> Peer de028c7533a3448da2355a31573994b0 (127.18.166.252:37123): Couldn't send request to peer de028c7533a3448da2355a31573994b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 05:16:01.583447 26199 raft_consensus.cc:2229] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:01.583554 26199 raft_consensus.cc:2258] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:01.583607 26199 tablet_replica.cc:330] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: stopping tablet replica
I20260812 05:16:01.587713 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.578816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.18.166.253:37805)
0812 05:16:01.578860 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.578863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.583418 (+ 4555us) server_negotiation.cc:407] Connection header received
0812 05:16:01.583540 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.583541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.583569 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.583612 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:01.583872 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.584573 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.586397 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.587115 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.587177 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.587255 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.587568 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.587620 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:16:01.588044 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.578767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44712)
0812 05:16:01.583351 (+ 4584us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.583365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.583456 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.583721 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.583722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.583724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:01.583784 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.583787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.584842 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.584843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.586253 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.586259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.587343 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.587345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.587432 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.587471 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.587881 (+ 410us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.587940 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4561}
I20260812 05:16:01.588478 26154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 LEARNER]: Refusing update from remote peer 4cbc350ebb29477ba381a76b5abd0ed7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 05:16:01.588586 26199 master.cc:552] Master@127.18.166.252:37123 shutdown complete.
I20260812 05:16:01.588639 26199 master_runner.cc:305] Clearing existing system tablet
I20260812 05:16:01.588799 26199 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.588858 26199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.588811 26104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 05:16:01.589577 26154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER de028c7533a3448da2355a31573994b0 (127.18.166.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.589953 26199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.590555 26311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.590721 26199 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.590847 26199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.590890 26199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
uuid: "de028c7533a3448da2355a31573994b0"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "dd278694bbfce5b4746d2b8bdace99b8"
server_key_iv: "87a33351c10cc01cd83b964dccef594a"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.591221 26184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cbc350ebb29477ba381a76b5abd0ed7. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.591334 26184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: LEARNER
I20260812 05:16:01.591389 26184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: NON_VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: true } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.591441 26184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: LEARNER
I20260812 05:16:01.591596 26104 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: attempting to promote NON_VOTER 25e23785489c43d0a39e77a995650c42 to VOTER
I20260812 05:16:01.591864 26104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } }
I20260812 05:16:01.592037 26199 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:01.593070 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.592912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34538)
0812 05:16:01.592965 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.593023 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:37123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 05:16:01.593521 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.593396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34552)
0812 05:16:01.593469 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.593494 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:37123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 05:16:01.593655 25491 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 -> Peer de028c7533a3448da2355a31573994b0 (127.18.166.252:37123): Couldn't send request to peer de028c7533a3448da2355a31573994b0. Status: Network error: Client connection negotiation failed: client connection to 127.18.166.252:37123: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 05:16:01.593839 26154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 LEARNER]: Refusing update from remote peer 4cbc350ebb29477ba381a76b5abd0ed7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 05:16:01.593959 26199 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 05:16:01.594079 26199 log.cc:1196] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 05:16:01.594151 26105 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 05:16:01.594405 26199 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Deleting consensus metadata
I20260812 05:16:01.594579 26199 master_runner.cc:315] Copying system tablet from 127.18.166.254:35127
I20260812 05:16:01.595037 26105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 25e23785489c43d0a39e77a995650c42 (127.18.166.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.595453 26199 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: tablet copy: Beginning tablet copy session from remote peer at address 127.18.166.254:35127
I20260812 05:16:01.596154 26105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.596248 26105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:01.596309 26105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.596367 26105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:01.596716 26154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 25e23785489c43d0a39e77a995650c42 (127.18.166.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.597751 26312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.597885 26312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:16:01.608855 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.601832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.18.166.254:35127)
0812 05:16:01.601893 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.601895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.602024 (+ 129us) server_negotiation.cc:407] Connection header received
0812 05:16:01.602118 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.602119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.602135 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.602205 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.602694 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.603361 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.604237 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.604522 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.604604 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.605938 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.605940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.605941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.605949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.607612 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.607986 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.607988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.607989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.608056 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.608279 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.608280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.608281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.608410 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.608486 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.608658 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.608747 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:16:01.608876 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.601790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45062)
0812 05:16:01.601952 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.601965 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.602047 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.602227 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.602228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.602457 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.602527 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.602529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.603443 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.603445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.604153 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.604156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.604604 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.604605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.604679 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.605282 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.605292 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.605839 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.607710 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.607711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.607711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.607909 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.608133 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.608134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.608134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.608209 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.608476 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.608477 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.608531 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.608618 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.608750 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":113,"threads_started":1}
I20260812 05:16:01.609604 25528 tablet_copy_service.cc:140] P 4cbc350ebb29477ba381a76b5abd0ed7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer de028c7533a3448da2355a31573994b0 ({username='oryx', principal='oryx/127.18.166.252@KRBTEST.COM'} at 127.0.0.1:45062)
I20260812 05:16:01.609957 25528 tablet_copy_service.cc:161] P 4cbc350ebb29477ba381a76b5abd0ed7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer de028c7533a3448da2355a31573994b0 at {username='oryx', principal='oryx/127.18.166.252@KRBTEST.COM'} at 127.0.0.1:45062: session id = de028c7533a3448da2355a31573994b0-00000000000000000000000000000000
I20260812 05:16:01.611904 25528 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 05:16:01.613085 26199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 05:16:01.614302 26199 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: tablet copy: Starting download of 0 data blocks...
I20260812 05:16:01.614476 26199 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: tablet copy: Starting download of 1 WAL segments...
I20260812 05:16:01.615288 26199 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 05:16:01.616109 25528 tablet_copy_service.cc:342] P 4cbc350ebb29477ba381a76b5abd0ed7: Request end of tablet copy session de028c7533a3448da2355a31573994b0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.166.252@KRBTEST.COM'} at 127.0.0.1:45062
I20260812 05:16:01.616197 25528 tablet_copy_service.cc:433] P 4cbc350ebb29477ba381a76b5abd0ed7: ending tablet copy session de028c7533a3448da2355a31573994b0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer de028c7533a3448da2355a31573994b0
I20260812 05:16:01.618791 26199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:01.620217 26320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.620179 26319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.620376 26199 server_base.cc:1034] running on GCE node
W20260812 05:16:01.620631 26322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.620790 26199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:01.620967 26199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:01.622107 26199 hybrid_clock.cc:648] HybridClock initialized: now 1786511761622097 us; error 28 us; skew 500 ppm
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:01.623587 26199 init.cc:377] Logged in from keytab as oryx/127.18.166.252@KRBTEST.COM (short username oryx)
I20260812 05:16:01.623919 26199 webserver.cc:460] Webserver started at http://127.18.166.252:42369/ using document root <none> and password file <none>
I20260812 05:16:01.624047 26199 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.624130 26199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.624806 26199 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:01.625773 26329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.625943 26199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.625993 26199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
uuid: "de028c7533a3448da2355a31573994b0"
format_stamp: "Formatted at 2026-08-12 05:16:01 on dist-test-slave-srgv"
server_key: "dd278694bbfce5b4746d2b8bdace99b8"
server_key_iv: "87a33351c10cc01cd83b964dccef594a"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.626055 26199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.632211 26199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:01.634244 26199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:01.637671 26199 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.252:37123
I20260812 05:16:01.638094 26199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
I20260812 05:16:01.638907 26384 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:16:01.639484 26384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Bootstrap starting.
I20260812 05:16:01.640836 26383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.252:37123 every 8 connection(s)
I20260812 05:16:01.642405 26384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:16:01.642644 26384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: Bootstrap complete.
I20260812 05:16:01.642905 26384 raft_consensus.cc:348] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } }
I20260812 05:16:01.643002 26384 raft_consensus.cc:729] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: de028c7533a3448da2355a31573994b0, State: Initialized, Role: LEARNER
I20260812 05:16:01.643049 26384 consensus_queue.cc:260] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } }
I20260812 05:16:01.643254 26384 sys_catalog.cc:564] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:01.643246 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.643551 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: This master's current role is: LEARNER
I20260812 05:16:01.644871 26399 catalog_manager.cc:1259] Loaded cluster ID: 116db59afc0442879118835f9a646747
I20260812 05:16:01.644966 26399 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: loading cluster ID for follower catalog manager: success
I20260812 05:16:01.646183 26399 catalog_manager.cc:1574] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: acquiring CA information for follower catalog manager: success
I20260812 05:16:01.646494 26399 catalog_manager.cc:1602] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 05:16:01.699081 26308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:01.699450 26308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:01.699501 26308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:01.701558 26308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:01.701650 26308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:01.701697 26308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:01.704389 26308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:36853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=46125
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:01.704756 26308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:01.705058 26308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:01.711539 26406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.711622 26308 server_base.cc:1034] running on GCE node
W20260812 05:16:01.711798 26409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.711534 26407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.712076 26308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:01.712423 26308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:01.713601 26308 hybrid_clock.cc:648] HybridClock initialized: now 1786511761713586 us; error 35 us; skew 500 ppm
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:01.717378 26308 init.cc:377] Logged in from keytab as oryx/127.18.166.193@KRBTEST.COM (short username oryx)
I20260812 05:16:01.718137 26308 webserver.cc:460] Webserver started at http://127.18.166.193:46125/ using document root <none> and password file <none>
I20260812 05:16:01.718513 26308 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.718612 26308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.720587 26308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:16:01.721330 26417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.721573 26308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:01.721701 26308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "5989a69309634f8b910185d08e18992d"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "c7ccddfb4a9b437c76ef0fbb08d14043"
server_key_iv: "3934db76cc49006fc9025ece6cb1ceef"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.722069 26308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.734898 26308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:01.737766 26308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:01.737911 26308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:01.738210 26308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:01.738572 26308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:01.738610 26308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.738634 26308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:01.738682 26308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.746768 26308 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:36853
I20260812 05:16:01.746820 26538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:36853 every 8 connection(s)
I20260812 05:16:01.747254 26308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:01.755615 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 26308
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.252@KRBTEST.COM
I20260812 05:16:01.755820 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25711
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:01.759996 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.748371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:44099 (local address 127.18.166.252:37123)
0812 05:16:01.748497 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.748500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.748927 (+ 427us) server_negotiation.cc:407] Connection header received
0812 05:16:01.750229 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.750230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.750247 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.750311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.751102 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.752131 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753119 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.753464 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753553 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.756519 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.756522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.756522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.756533 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.758339 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.758856 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.758857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.758858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.758931 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.759224 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.759225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.759225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.759373 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.759455 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.759774 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.759891 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
I20260812 05:16:01.760210 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.748132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.18.166.193:44099)
0812 05:16:01.748842 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.748851 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.750044 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.750378 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.750379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.750804 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.750982 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.750989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.752223 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.752225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.753033 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.753035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.753552 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.753643 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.754426 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.754432 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.756351 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.758452 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.758466 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.758470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.758747 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.759006 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.759008 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.759008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.759150 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.759446 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.759455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.759623 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.759741 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.759933 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":65,"thread_start_us":108,"threads_started":1}
I20260812 05:16:01.760859 26539 heartbeater.cc:347] Connected to a master server at 127.18.166.252:37123
I20260812 05:16:01.760990 26539 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:01.761197 26539 heartbeater.cc:511] Master 127.18.166.252:37123 requested a full tablet report, sending...
I20260812 05:16:01.761839 26346 ts_manager.cc:194] Registered new tserver with Master: 5989a69309634f8b910185d08e18992d (127.18.166.193:36853)
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
I20260812 05:16:01.766601 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:38983
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:35127
--webserver_port=37999
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:16:01.768461 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.748517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:58455 (local address 127.18.166.254:35127)
0812 05:16:01.748581 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.748583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.748731 (+ 148us) server_negotiation.cc:407] Connection header received
0812 05:16:01.750167 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.750169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.750185 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.750250 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.751095 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.751640 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753082 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.753350 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753397 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.765256 (+ 11859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.765259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.765260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.765275 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.767039 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.767467 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.767468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.767469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.767555 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.767781 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.767782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.767782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.767917 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.767996 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.768219 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.768329 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:16:01.769011 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.748430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:53717 (local address 127.18.166.253:37805)
0812 05:16:01.748509 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.748512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.748741 (+ 229us) server_negotiation.cc:407] Connection header received
0812 05:16:01.750231 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.750232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.750255 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.750311 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.751103 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.751879 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.752926 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.753203 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753273 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.765279 (+ 12006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.765281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.765281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.765291 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.767396 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.768029 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.768031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.768031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.768103 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.768313 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.768314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.768314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.768445 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.768527 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.768824 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.768914 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 05:16:01.769040 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.748133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.18.166.193:53717)
0812 05:16:01.748570 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.748624 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.750012 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.750380 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.750381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.750772 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.750982 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.750989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.751979 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.751980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.752816 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.752818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.753305 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.753511 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.754349 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.754362 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.765138 (+ 10776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.767666 (+ 2528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.767668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.767668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.767933 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.768186 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.768187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.768187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.768255 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.768640 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.768641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.768698 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.768770 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.768930 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":8,"thread_start_us":124,"threads_started":1}
I20260812 05:16:01.769104 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.748132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:58455)
0812 05:16:01.748581 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.748624 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.750012 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.750346 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.750351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.750740 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.750982 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.750986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.751803 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.751804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.752960 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.752963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.753513 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.753514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.753585 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.754368 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.754372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.765118 (+ 10746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.767168 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.767170 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.767171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.767373 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.767639 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.767640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.767641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.767709 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.767997 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.767999 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.768055 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.768628 (+ 573us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.768775 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":6,"thread_start_us":129,"threads_started":1}
I20260812 05:16:01.769755 26541 heartbeater.cc:347] Connected to a master server at 127.18.166.253:37805
I20260812 05:16:01.769752 26540 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35127
I20260812 05:16:01.769869 26541 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:01.769879 26540 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:01.769980 26541 heartbeater.cc:511] Master 127.18.166.253:37805 requested a full tablet report, sending...
I20260812 05:16:01.769981 26540 heartbeater.cc:511] Master 127.18.166.254:35127 requested a full tablet report, sending...
I20260812 05:16:01.770315 25508 ts_manager.cc:194] Re-registered known tserver with Master: 5989a69309634f8b910185d08e18992d (127.18.166.193:36853)
I20260812 05:16:01.770399 26144 ts_manager.cc:194] Registered new tserver with Master: 5989a69309634f8b910185d08e18992d (127.18.166.193:36853)
I20260812 05:16:01.770840 25508 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:58455
W20260812 05:16:01.857481 26548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:01.857725 26548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:01.857754 26548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:01.859047 26548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:01.859107 26548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:01.859141 26548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:01.860867 26548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:38983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=37999
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:01.861181 26548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:01.861526 26548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:01.867002 26557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.867029 26553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:01.867002 26554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:01.867311 26548 server_base.cc:1034] running on GCE node
I20260812 05:16:01.867549 26548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:01.867815 26548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:01.868996 26548 hybrid_clock.cc:648] HybridClock initialized: now 1786511761868985 us; error 46 us; skew 500 ppm
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:01.871919 26548 init.cc:377] Logged in from keytab as oryx/127.18.166.194@KRBTEST.COM (short username oryx)
I20260812 05:16:01.872442 26548 webserver.cc:460] Webserver started at http://127.18.166.194:37999/ using document root <none> and password file <none>
I20260812 05:16:01.872675 26548 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:01.872740 26548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:01.874228 26548 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:01.874863 26564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.875058 26548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:01.875160 26548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "00365876de384b3b8745214b55861708"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "cb4a0ba17e49290e7ed9684951eaa248"
server_key_iv: "8644ffebd37afe5283dcf75554d89aea"
server_key_version: "encryptionkey@0"
I20260812 05:16:01.875538 26548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:01.887198 26548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:01.889668 26548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:01.889812 26548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:01.890029 26548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:01.890396 26548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:01.890435 26548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.890460 26548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:01.890472 26548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:01.897013 26548 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38983
I20260812 05:16:01.897119 26681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38983 every 8 connection(s)
I20260812 05:16:01.897583 26548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:01.903844 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 26548
I20260812 05:16:01.904032 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25852
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.252@KRBTEST.COM
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
Aug 12 05:16:01 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511761, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:01.916558 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:46791
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:35127
--webserver_port=36579
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:16:01.920317 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.898539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:54125 (local address 127.18.166.253:37805)
0812 05:16:01.898597 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.898599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.898801 (+ 202us) server_negotiation.cc:407] Connection header received
0812 05:16:01.899506 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.899508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.899521 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.899555 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.900570 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.901357 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902510 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.902801 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902883 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.916589 (+ 13706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.916591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.916591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.916601 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.918423 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919026 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.919028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.919029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.919093 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919439 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.919440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.919440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.919604 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.919655 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.920084 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.920186 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:16:01.920720 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.898543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:46335 (local address 127.18.166.252:37123)
0812 05:16:01.898747 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.898751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.898835 (+ 84us) server_negotiation.cc:407] Connection header received
0812 05:16:01.899497 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.899498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.899515 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.899584 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.900561 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.901124 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902945 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.903204 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.903277 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.916204 (+ 12927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.916206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.916206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.916219 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.919166 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919666 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.919668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.919668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.919736 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919962 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.919963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.919963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.920134 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.920202 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.920476 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.920575 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260812 05:16:01.920816 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.898471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:37379)
0812 05:16:01.898910 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.898920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.899392 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.899698 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.899702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.900313 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.900455 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.900461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.901471 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.901472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.902255 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.902257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.902733 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.902867 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.903712 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.903716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.916808 (+ 13092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.919316 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.919317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.919610 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.919841 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.919842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.919900 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.920191 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.920192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.920232 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.920307 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.920524 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":189,"thread_start_us":82,"threads_started":1}
I20260812 05:16:01.920816 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.898471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.18.166.194:54125)
0812 05:16:01.898663 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.898707 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.899379 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.899697 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.899702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.900304 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.900455 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.900461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.901513 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.901514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.902433 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.902435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.902887 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.902944 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.903678 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.903690 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.916457 (+ 12767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.918520 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.918536 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.918539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.918909 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.919175 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.919176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.919353 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.919707 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.919717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.919914 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.920427 (+ 513us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.920574 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":62,"thread_start_us":108,"threads_started":1}
I20260812 05:16:01.921063 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.898750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:37379 (local address 127.18.166.254:35127)
0812 05:16:01.898830 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.898832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.898976 (+ 144us) server_negotiation.cc:407] Connection header received
0812 05:16:01.899506 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.899508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.899521 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.899555 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:01.900571 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.901335 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902363 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.902645 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.902715 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.916973 (+ 14258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:01.916975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:01.916975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:01.916985 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:01.919225 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919697 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.919699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.919699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.919761 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919966 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:01.919967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:01.919968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:01.920103 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:01.920163 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.920882 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.920977 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 05:16:01.921319 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.898470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.18.166.194:46335)
0812 05:16:01.898764 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.898775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.899379 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.899997 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.899998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.900241 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:01.900453 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.900458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.901248 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.901249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.902416 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.902419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.903283 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.903284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.903330 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.903873 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:01.903877 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:01.916002 (+ 12125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:01.919276 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.919278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.919557 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.919820 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:01.919820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:01.919821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:01.919879 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:01.920233 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:01.920234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:01.920289 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.920858 (+ 569us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.921177 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":11,"thread_start_us":130,"threads_started":1}
I20260812 05:16:01.921572 26684 heartbeater.cc:347] Connected to a master server at 127.18.166.253:37805
I20260812 05:16:01.921682 26684 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:01.921726 26682 heartbeater.cc:347] Connected to a master server at 127.18.166.252:37123
I20260812 05:16:01.921757 26683 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35127
I20260812 05:16:01.921828 26683 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:01.921762 26682 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:01.921919 26684 heartbeater.cc:511] Master 127.18.166.253:37805 requested a full tablet report, sending...
I20260812 05:16:01.921933 26683 heartbeater.cc:511] Master 127.18.166.254:35127 requested a full tablet report, sending...
I20260812 05:16:01.921984 26682 heartbeater.cc:511] Master 127.18.166.252:37123 requested a full tablet report, sending...
I20260812 05:16:01.922344 25508 ts_manager.cc:194] Re-registered known tserver with Master: 00365876de384b3b8745214b55861708 (127.18.166.194:38983)
I20260812 05:16:01.922405 26346 ts_manager.cc:194] Registered new tserver with Master: 00365876de384b3b8745214b55861708 (127.18.166.194:38983)
I20260812 05:16:01.922497 26144 ts_manager.cc:194] Registered new tserver with Master: 00365876de384b3b8745214b55861708 (127.18.166.194:38983)
I20260812 05:16:01.922864 25508 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:37379
I20260812 05:16:01.993937 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.991306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.18.166.252:37123)
0812 05:16:01.991384 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:01.991386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:01.991396 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:01.991429 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:01.991430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:01.991463 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:01.991542 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:01.991686 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.992186 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.993097 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.993576 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.993670 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.993753 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:01.993815 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 05:16:01.993856 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:16:01.993937 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:01.991217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34568)
0812 05:16:01.991290 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:01.991301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:01.991370 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:01.991547 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:01.991548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:01.991550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:01.991615 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.991618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.992276 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.992277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.992994 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:01.992996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:01.993670 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:01.993671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:01.993737 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:01.993741 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:01.993808 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:16:01.993857 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:16:01.994280 26356 raft_consensus.cc:1203] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 05:16:01.994442 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cbc350ebb29477ba381a76b5abd0ed7. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:01.994561 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: This master's current role is: LEARNER
W20260812 05:16:02.010026 26689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:02.010255 26689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:02.010289 26689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:02.011443 26689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:02.011480 26689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:02.011507 26689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:02.012984 26689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:46791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=36579
--tserver_master_addrs=127.18.166.254:35127,127.18.166.253:37805,127.18.166.252:37123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:02.013190 26689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:02.013399 26689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:02.018770 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:02.018816 26697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:02.018784 26695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:02.018996 26689 server_base.cc:1034] running on GCE node
I20260812 05:16:02.019214 26689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:02.019425 26689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:02.020602 26689 hybrid_clock.cc:648] HybridClock initialized: now 1786511762020594 us; error 35 us; skew 500 ppm
Aug 12 05:16:02 dist-test-slave-srgv krb5kdc[25461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511762, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:02.023404 26689 init.cc:377] Logged in from keytab as oryx/127.18.166.195@KRBTEST.COM (short username oryx)
I20260812 05:16:02.023898 26689 webserver.cc:460] Webserver started at http://127.18.166.195:36579/ using document root <none> and password file <none>
I20260812 05:16:02.024185 26689 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:02.024293 26689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:02.026041 26689 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:16:02.026729 26704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:02.026904 26689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:02.027040 26689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "d72cb9c007d54c6ca0150247f1251ea0"
format_stamp: "Formatted at 2026-08-12 05:16:00 on dist-test-slave-srgv"
server_key: "12ad673c9c4913cb869c88ff9e202020"
server_key_iv: "8d61498da0b434a3b259f409c6ffd899"
server_key_version: "encryptionkey@0"
I20260812 05:16:02.027431 26689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:02.061614 26689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:02.064075 26689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:02.064265 26689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:02.064528 26689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:02.064868 26689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:02.064924 26689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:02.064960 26689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:02.064999 26689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:02.071617 26689 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:46791
I20260812 05:16:02.071661 26821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:46791 every 8 connection(s)
I20260812 05:16:02.072042 26689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:02.076349 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 26689
Aug 12 05:16:02 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511762, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.252@KRBTEST.COM
Aug 12 05:16:02 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511762, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
Aug 12 05:16:02 dist-test-slave-srgv krb5kdc[25461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511762, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.253@KRBTEST.COM
I20260812 05:16:02.086037 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.073145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:33385 (local address 127.18.166.252:37123)
0812 05:16:02.073203 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.073205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.073574 (+ 369us) server_negotiation.cc:407] Connection header received
0812 05:16:02.074739 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.074740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.074754 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.074822 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.075481 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.075989 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077522 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.077840 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077884 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.082392 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.082394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.082394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.082402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.084249 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.084780 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.084781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.084781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.084843 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.085150 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.085151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.085151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.085301 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.085359 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.085812 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.085906 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 05:16:02.086654 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.072963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.18.166.195:33385)
0812 05:16:02.073511 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.073521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.074415 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.074827 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.074829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.075171 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:02.075376 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.075380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.076105 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.076106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.077398 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.077400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.077974 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.078130 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.078958 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:02.078968 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:02.082293 (+ 3325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:02.084342 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:02.084353 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:02.084356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:02.084688 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:02.084912 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:02.084913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:02.084913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:02.085063 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:02.085400 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:02.085433 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:02.085638 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.086153 (+ 515us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.086395 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":329,"thread_start_us":75,"threads_started":1}
I20260812 05:16:02.087260 26822 heartbeater.cc:347] Connected to a master server at 127.18.166.252:37123
I20260812 05:16:02.087383 26822 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:02.087533 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.073265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:45489 (local address 127.18.166.253:37805)
0812 05:16:02.073348 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.073350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.073820 (+ 470us) server_negotiation.cc:407] Connection header received
0812 05:16:02.074549 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.074550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.074564 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.074634 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.075493 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.076285 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.078319 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.078630 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.078686 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.082562 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.082564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.082564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.082573 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.085394 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.086158 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.086160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.086160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.086229 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.086753 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.086754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.086755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.086895 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.086951 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.087330 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.087414 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 05:16:02.087615 26822 heartbeater.cc:511] Master 127.18.166.252:37123 requested a full tablet report, sending...
I20260812 05:16:02.088045 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.072963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.18.166.195:45489)
0812 05:16:02.073727 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.073736 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.074429 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.074743 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.074747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.075221 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:02.075376 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.075382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.077313 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.078211 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.078214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.078759 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.078760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.078812 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.079424 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:02.079428 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:02.082469 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:02.085635 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:02.085636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:02.085636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:02.085857 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:02.086605 (+ 748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:02.086606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:02.086606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:02.086667 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:02.086987 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:02.086988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:02.087031 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.087803 (+ 772us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.087931 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":484,"thread_start_us":90,"threads_started":1}
I20260812 05:16:02.088140 26346 ts_manager.cc:194] Registered new tserver with Master: d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:02.088387 26824 heartbeater.cc:347] Connected to a master server at 127.18.166.253:37805
I20260812 05:16:02.088446 26824 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:02.088527 26824 heartbeater.cc:511] Master 127.18.166.253:37805 requested a full tablet report, sending...
I20260812 05:16:02.088807 26144 ts_manager.cc:194] Registered new tserver with Master: d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:02.089728 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.073144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:36899 (local address 127.18.166.254:35127)
0812 05:16:02.073208 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.073210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.073332 (+ 122us) server_negotiation.cc:407] Connection header received
0812 05:16:02.074609 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.074612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.074624 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.074681 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.075494 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.076280 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077533 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.077841 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077888 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.082909 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.082910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.082911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.082919 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.086040 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.087156 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.087157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.087158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.087216 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.089051 (+ 1835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.089052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.089053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.089226 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.089285 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.089521 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.089618 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 05:16:02.090444 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.079032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.18.166.254:35127)
0812 05:16:02.079262 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.079264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.080376 (+ 1112us) server_negotiation.cc:407] Connection header received
0812 05:16:02.080479 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.080480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.080494 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.080538 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.081312 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.082163 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.083211 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.083571 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.083621 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.086815 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.086817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.086817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.086824 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.088690 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.089241 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.089242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.089242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.089311 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.089573 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.089574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.089575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.089731 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.089827 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.090226 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.090327 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":126,"threads_started":1}
I20260812 05:16:02.090452 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.072963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:36899)
0812 05:16:02.073169 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.073215 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.074412 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.074743 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.074747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.075185 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:02.075376 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.075382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.076381 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.076382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.077398 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.077402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.077977 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.077978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.078129 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.079022 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:02.079028 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:02.082176 (+ 3148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:02.086299 (+ 4123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:02.086300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:02.086301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:02.086532 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:02.087685 (+ 1153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:02.087686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:02.087687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:02.087752 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:02.089326 (+ 1574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:02.089327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:02.089375 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.089859 (+ 484us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.089992 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260812 05:16:02.090916 26823 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35127
I20260812 05:16:02.090977 26823 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:02.091059 26823 heartbeater.cc:511] Master 127.18.166.254:35127 requested a full tablet report, sending...
I20260812 05:16:02.091149 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.078148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.18.166.253:37805)
0812 05:16:02.078416 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.078418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.079552 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 05:16:02.079577 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.079577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.079600 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.079658 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.080553 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.081565 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.082663 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.084073 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.084142 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.085743 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.085745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.085745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.085752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.089454 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.089915 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.089916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.089916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.089979 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.090186 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.090187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.090187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.090331 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.090435 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.090947 (+ 512us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.091041 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":61,"threads_started":1}
I20260812 05:16:02.091354 25508 ts_manager.cc:194] Re-registered known tserver with Master: d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:02.091854 25508 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:36899
I20260812 05:16:02.094317 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.077890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.18.166.252:37123)
0812 05:16:02.079898 (+ 2008us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.079900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.080130 (+ 230us) server_negotiation.cc:407] Connection header received
0812 05:16:02.080156 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.080157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.080168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.080231 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.080922 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.081674 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.086363 (+ 4689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.086650 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.086721 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.090255 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.090257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.090257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.090265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.093044 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.093527 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.093529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.093529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.093605 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.093807 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.093808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.093809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.093945 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.094043 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.094131 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.094221 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1980,"thread_start_us":57,"threads_started":1}
I20260812 05:16:02.092808 25508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:16:02.095281 25508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:02.095999 26104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: NON_VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: true } } }
I20260812 05:16:02.096128 26104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:02.097456 26104 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: attempting to promote NON_VOTER de028c7533a3448da2355a31573994b0 to VOTER
I20260812 05:16:02.097594 26104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } }
I20260812 05:16:02.097963 26154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Refusing update from remote peer 4cbc350ebb29477ba381a76b5abd0ed7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 05:16:02.098004 26356 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 LEARNER]: Refusing update from remote peer 4cbc350ebb29477ba381a76b5abd0ed7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 05:16:02.098171 26101 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 05:16:02.098248 26103 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 05:16:02.098829 26101 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, de028c7533a3448da2355a31573994b0 (127.18.166.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.099205 26356 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, de028c7533a3448da2355a31573994b0 (127.18.166.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.100031 26101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.100093 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.100222 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:02.100219 26154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, de028c7533a3448da2355a31573994b0 (127.18.166.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.100274 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.100332 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:02.100459 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.100543 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:16:02.100126 26101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:02.101495 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:02.101598 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:16:02.106937 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.101574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38983 (local address 127.0.0.1:36670)
0812 05:16:02.101739 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.101752 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.101800 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.102495 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.102496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.102498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:02.102553 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.102555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.104356 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.104357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.105646 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.105649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.106674 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.106674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.106751 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.106755 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.106811 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.106852 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":36}
I20260812 05:16:02.106976 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.101785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.18.166.194:38983)
0812 05:16:02.102324 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.102330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.102343 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:16:02.102389 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.102391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.102409 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.102500 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:02.102624 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.104235 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.105745 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.106581 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.106690 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.106762 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.106827 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.106872 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":100,"threads_started":1}
I20260812 05:16:02.107764 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.102223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:46791 (local address 127.0.0.1:39928)
0812 05:16:02.102385 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.102398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.102459 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.104587 (+ 2128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.104588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.104591 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:02.104641 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.104643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.105535 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.105536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.106664 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.106668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.107510 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.107511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.107572 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.107576 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.107632 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.107678 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260812 05:16:02.107877 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.101635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:36853 (local address 127.0.0.1:37226)
0812 05:16:02.102112 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.102123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.102174 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.103012 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.103013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.103015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:02.103052 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.103054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.104380 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.104380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.105646 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.105660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.107668 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.107669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.107724 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.107727 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.107763 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.107809 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":62,"thread_start_us":98,"threads_started":1}
I20260812 05:16:02.107990 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.101879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.18.166.193:36853)
0812 05:16:02.102829 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.102834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.102845 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:02.102894 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.102896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.102914 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.103000 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:02.103149 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.104263 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.106805 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.107587 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.107693 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.107749 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.107788 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.107910 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888,"thread_start_us":104,"threads_started":1}
I20260812 05:16:02.108148 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.102494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.18.166.195:46791)
0812 05:16:02.104424 (+ 1930us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.104429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.104437 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:02.104471 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.104473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.104486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.104586 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:02.104700 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.105373 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.106752 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.107406 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.107534 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.107595 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.107996 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.108047 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1875,"thread_start_us":105,"threads_started":1}
I20260812 05:16:02.108258 26614 tablet_service.cc:1467] Processing CreateTablet for tablet 8f341b36ccb34b41a9314ca58f6c352c (DEFAULT_TABLE table=test-table [id=6d4e939505b940be9cd53e70ea99248c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:02.108673 26614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f341b36ccb34b41a9314ca58f6c352c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:02.108760 26468 tablet_service.cc:1467] Processing CreateTablet for tablet 8f341b36ccb34b41a9314ca58f6c352c (DEFAULT_TABLE table=test-table [id=6d4e939505b940be9cd53e70ea99248c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:02.108774 26755 tablet_service.cc:1467] Processing CreateTablet for tablet 8f341b36ccb34b41a9314ca58f6c352c (DEFAULT_TABLE table=test-table [id=6d4e939505b940be9cd53e70ea99248c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:02.109032 26468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f341b36ccb34b41a9314ca58f6c352c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:02.109119 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f341b36ccb34b41a9314ca58f6c352c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:02.112581 26855 tablet_bootstrap.cc:492] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Bootstrap starting.
I20260812 05:16:02.112751 26856 tablet_bootstrap.cc:492] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Bootstrap starting.
I20260812 05:16:02.112952 26854 tablet_bootstrap.cc:492] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Bootstrap starting.
I20260812 05:16:02.113597 26855 tablet_bootstrap.cc:654] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:02.113597 26856 tablet_bootstrap.cc:654] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:02.113679 26854 tablet_bootstrap.cc:654] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:02.113976 26856 log.cc:824] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:02.113992 26854 log.cc:824] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:02.114004 26855 log.cc:824] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:02.114903 26854 tablet_bootstrap.cc:492] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: No bootstrap required, opened a new log
I20260812 05:16:02.114984 26854 ts_tablet_manager.cc:1397] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:16:02.115072 26855 tablet_bootstrap.cc:492] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: No bootstrap required, opened a new log
I20260812 05:16:02.115149 26855 ts_tablet_manager.cc:1397] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:16:02.116884 26854 raft_consensus.cc:348] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.117028 26854 raft_consensus.cc:374] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:02.117053 26854 raft_consensus.cc:729] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5989a69309634f8b910185d08e18992d, State: Initialized, Role: FOLLOWER
I20260812 05:16:02.116995 26855 raft_consensus.cc:348] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.117190 26855 raft_consensus.cc:374] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:02.117195 26854 consensus_queue.cc:260] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.117241 26855 raft_consensus.cc:729] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00365876de384b3b8745214b55861708, State: Initialized, Role: FOLLOWER
I20260812 05:16:02.117358 26855 consensus_queue.cc:260] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.117530 26854 ts_tablet_manager.cc:1428] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:16:02.117691 26540 heartbeater.cc:503] Master 127.18.166.254:35127 was elected leader, sending a full tablet report...
I20260812 05:16:02.117862 26856 tablet_bootstrap.cc:492] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: No bootstrap required, opened a new log
I20260812 05:16:02.117942 26856 ts_tablet_manager.cc:1397] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260812 05:16:02.118631 26855 ts_tablet_manager.cc:1428] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:16:02.119444 26683 heartbeater.cc:503] Master 127.18.166.254:35127 was elected leader, sending a full tablet report...
I20260812 05:16:02.120270 26856 raft_consensus.cc:348] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.120448 26856 raft_consensus.cc:374] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:02.120500 26856 raft_consensus.cc:729] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72cb9c007d54c6ca0150247f1251ea0, State: Initialized, Role: FOLLOWER
I20260812 05:16:02.120613 26856 consensus_queue.cc:260] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.120916 26856 ts_tablet_manager.cc:1428] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:16:02.120977 26823 heartbeater.cc:503] Master 127.18.166.254:35127 was elected leader, sending a full tablet report...
W20260812 05:16:02.149317 26685 tablet.cc:2369] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:16:02.248751 26542 tablet.cc:2369] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:16:02.318876 26860 raft_consensus.cc:482] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:02.319051 26860 raft_consensus.cc:504] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.319458 26860 leader_election.cc:288] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00365876de384b3b8745214b55861708 (127.18.166.194:38983), d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:02.322643 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.319859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:51137 (local address 127.18.166.195:46791)
0812 05:16:02.319910 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.319912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.319979 (+ 67us) server_negotiation.cc:407] Connection header received
0812 05:16:02.320051 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.320052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.320109 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.320163 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:02.320350 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.320808 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.321889 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.322271 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.322330 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.322336 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.322519 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.322565 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:16:02.322669 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.319828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:47131 (local address 127.18.166.194:38983)
0812 05:16:02.319894 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.319898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.319936 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:16:02.320015 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.320016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.320027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.320089 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:02.320265 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.320799 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.321891 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.322291 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.322389 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.322398 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.322552 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.322597 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:16:02.322700 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.319696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:46791 (local address 127.18.166.193:51137)
0812 05:16:02.319866 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.319879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.319972 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.320185 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.320186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.320188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:02.320270 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.320272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.320921 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.320922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.321790 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.321792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.322356 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.322357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.322435 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.322445 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.322491 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.322559 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":29,"thread_start_us":72,"threads_started":1}
I20260812 05:16:02.322719 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.319695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38983 (local address 127.18.166.193:47131)
0812 05:16:02.319861 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:02.319874 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:02.319947 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:02.320109 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:02.320110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:02.320113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:02.320178 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.320179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.320896 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.320897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.321785 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:02.321788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.322407 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.322408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:02.322455 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.322458 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:02.322533 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:16:02.322582 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":72,"thread_start_us":65,"threads_started":1}
I20260812 05:16:02.323263 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f341b36ccb34b41a9314ca58f6c352c" candidate_uuid: "5989a69309634f8b910185d08e18992d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72cb9c007d54c6ca0150247f1251ea0" is_pre_election: true
I20260812 05:16:02.323259 26635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f341b36ccb34b41a9314ca58f6c352c" candidate_uuid: "5989a69309634f8b910185d08e18992d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00365876de384b3b8745214b55861708" is_pre_election: true
I20260812 05:16:02.323437 26775 raft_consensus.cc:2454] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5989a69309634f8b910185d08e18992d in term 0.
I20260812 05:16:02.323442 26635 raft_consensus.cc:2454] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5989a69309634f8b910185d08e18992d in term 0.
I20260812 05:16:02.323676 26418 leader_election.cc:302] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989a69309634f8b910185d08e18992d, d72cb9c007d54c6ca0150247f1251ea0; no voters:
I20260812 05:16:02.323818 26860 raft_consensus.cc:2781] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:02.323879 26860 raft_consensus.cc:482] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:02.323904 26860 raft_consensus.cc:3037] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Advancing to term 1
W20260812 05:16:02.324139 26825 tablet.cc:2369] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:16:02.324676 26860 raft_consensus.cc:504] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.324834 26860 leader_election.cc:288] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [CANDIDATE]: Term 1 election: Requested vote from peers 00365876de384b3b8745214b55861708 (127.18.166.194:38983), d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:02.325006 26635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f341b36ccb34b41a9314ca58f6c352c" candidate_uuid: "5989a69309634f8b910185d08e18992d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00365876de384b3b8745214b55861708"
I20260812 05:16:02.325071 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f341b36ccb34b41a9314ca58f6c352c" candidate_uuid: "5989a69309634f8b910185d08e18992d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72cb9c007d54c6ca0150247f1251ea0"
I20260812 05:16:02.325155 26635 raft_consensus.cc:3037] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:02.325176 26775 raft_consensus.cc:3037] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:02.326257 26635 raft_consensus.cc:2454] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5989a69309634f8b910185d08e18992d in term 1.
I20260812 05:16:02.326257 26775 raft_consensus.cc:2454] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5989a69309634f8b910185d08e18992d in term 1.
I20260812 05:16:02.326579 26418 leader_election.cc:302] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5989a69309634f8b910185d08e18992d, d72cb9c007d54c6ca0150247f1251ea0; no voters:
I20260812 05:16:02.326701 26860 raft_consensus.cc:2781] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:02.326823 26860 raft_consensus.cc:686] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 1 LEADER]: Becoming Leader. State: Replica: 5989a69309634f8b910185d08e18992d, State: Running, Role: LEADER
I20260812 05:16:02.327044 26860 consensus_queue.cc:237] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } }
I20260812 05:16:02.328233 25508 catalog_manager.cc:5495] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5989a69309634f8b910185d08e18992d (127.18.166.193). New cstate: current_term: 1 leader_uuid: "5989a69309634f8b910185d08e18992d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } health_report { overall_health: HEALTHY } } }
I20260812 05:16:02.334177 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.331858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.18.166.193:36853)
0812 05:16:02.331903 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.331905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.331917 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:16:02.331992 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.331992 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.332010 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.332069 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:02.332268 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.332732 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.333535 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.333730 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.333788 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.333919 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:02.334015 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:02.334070 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.334101 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.334117 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:16:02.336257 26635 raft_consensus.cc:1261] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Refusing update from remote peer 5989a69309634f8b910185d08e18992d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:02.336647 26775 raft_consensus.cc:1261] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Refusing update from remote peer 5989a69309634f8b910185d08e18992d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:02.336645 26860 consensus_queue.cc:1035] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [LEADER]: Connected to new peer: Peer: permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:02.336911 26860 consensus_queue.cc:1035] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:02.339890 26869 mvcc.cc:205] Tried to move back new op lower bound from 7317552178526228480 to 7317552178492346368. Current Snapshot: MvccSnapshot[applied={T|T < 7317552178526228480}]
I20260812 05:16:02.340029 26870 mvcc.cc:205] Tried to move back new op lower bound from 7317552178526228480 to 7317552178492346368. Current Snapshot: MvccSnapshot[applied={T|T < 7317552178526228480}]
I20260812 05:16:02.343892 25506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:02.344035 25506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:02.345556 25506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cbc350ebb29477ba381a76b5abd0ed7: Sending DeleteTablet for 3 replicas of tablet 8f341b36ccb34b41a9314ca58f6c352c
I20260812 05:16:02.345947 26755 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f341b36ccb34b41a9314ca58f6c352c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:02 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:39928
I20260812 05:16:02.346005 26614 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f341b36ccb34b41a9314ca58f6c352c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:02 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:36670
I20260812 05:16:02.346267 26880 tablet_replica.cc:330] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: stopping tablet replica
I20260812 05:16:02.346364 26881 tablet_replica.cc:330] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: stopping tablet replica
I20260812 05:16:02.346387 26880 raft_consensus.cc:2229] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:02.346493 26881 raft_consensus.cc:2229] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:02.346508 26880 raft_consensus.cc:2258] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:02.346602 26881 raft_consensus.cc:2258] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:02.347070 26880 ts_tablet_manager.cc:1905] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:02.347071 26881 ts_tablet_manager.cc:1905] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:02.348181 26468 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f341b36ccb34b41a9314ca58f6c352c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:02 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:37226
I20260812 05:16:02.348555 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 25474
I20260812 05:16:02.349063 26880 ts_tablet_manager.cc:1918] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:02.349149 26880 log.cc:1196] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/8f341b36ccb34b41a9314ca58f6c352c
I20260812 05:16:02.349552 26880 ts_tablet_manager.cc:1939] T 8f341b36ccb34b41a9314ca58f6c352c P d72cb9c007d54c6ca0150247f1251ea0: Deleting consensus metadata
I20260812 05:16:02.349823 26881 ts_tablet_manager.cc:1918] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:02.349893 26881 log.cc:1196] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/8f341b36ccb34b41a9314ca58f6c352c
I20260812 05:16:02.350126 26881 ts_tablet_manager.cc:1939] T 8f341b36ccb34b41a9314ca58f6c352c P 00365876de384b3b8745214b55861708: Deleting consensus metadata
I20260812 05:16:02.354151 26883 tablet_replica.cc:330] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: stopping tablet replica
I20260812 05:16:02.354328 26883 raft_consensus.cc:2229] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:02.354498 26883 raft_consensus.cc:2258] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:02.355058 26883 ts_tablet_manager.cc:1905] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 05:16:02.356540 26706 connection.cc:535] server connection from 127.0.0.1:39928 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:16:02.357204 26883 ts_tablet_manager.cc:1918] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:02.357319 26883 log.cc:1196] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/8f341b36ccb34b41a9314ca58f6c352c
W20260812 05:16:02.357331 26567 connection.cc:535] server connection from 127.0.0.1:36670 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:16:02.357694 26883 ts_tablet_manager.cc:1939] T 8f341b36ccb34b41a9314ca58f6c352c P 5989a69309634f8b910185d08e18992d: Deleting consensus metadata
W20260812 05:16:02.357695 26130 connection.cc:535] server connection from 127.0.0.1:44712 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 05:16:02.358222 26420 connection.cc:441] server connection from 127.0.0.1:37226 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37226 (request call id 1) could send its response
I20260812 05:16:02.362466 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.347585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.18.166.252:37123)
0812 05:16:02.347641 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.347644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.354011 (+ 6367us) server_negotiation.cc:407] Connection header received
0812 05:16:02.354051 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.354052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.354072 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.354146 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.355022 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.355817 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.356922 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.357238 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.357285 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.359081 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.359083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.359083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.359093 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.361227 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.361653 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.361654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.361654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.361716 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.361935 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.361936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.361937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.362052 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.362157 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.362269 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.362356 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:16:02.367101 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.359903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.18.166.252:37123)
0812 05:16:02.359970 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.359972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.360069 (+ 97us) server_negotiation.cc:407] Connection header received
0812 05:16:02.360114 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.360115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.360125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.360170 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.360835 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.361345 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.362854 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.363065 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.363118 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.364523 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.364524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.364524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.364531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.366039 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.366425 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.366426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.366426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.366475 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.366668 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.366668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.366668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.366768 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.366841 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.366980 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.367037 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:16:02.374006 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:02.360285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44736 (local address 127.18.166.253:37805)
0812 05:16:02.360327 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:02.360330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:02.367225 (+ 6895us) server_negotiation.cc:407] Connection header received
0812 05:16:02.367330 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:02.367332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:02.367347 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:02.367379 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:02.368072 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.368554 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.369569 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:02.369821 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:02.369904 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:02.371334 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:02.371335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:02.371336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:02.371343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:02.372788 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.373184 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.373185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.373185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.373240 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:02.373506 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:02.373506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:02.373507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:02.373641 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:02.373728 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:02.373865 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 05:16:02.373938 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:16:03.341545 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.341227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:38135)
0812 05:16:03.341376 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.341486 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260812 05:16:03.341704 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.341273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:47615)
0812 05:16:03.341560 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.341636 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":60,"threads_started":1}
W20260812 05:16:03.341807 26823 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:35127 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
W20260812 05:16:03.341921 26683 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:35127 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.342271 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.342023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:41039)
0812 05:16:03.342159 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.342227 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":49,"threads_started":1}
W20260812 05:16:03.342437 26540 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:35127 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.846992 26893 raft_consensus.cc:482] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4cbc350ebb29477ba381a76b5abd0ed7)
I20260812 05:16:03.847206 26893 raft_consensus.cc:504] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } }
I20260812 05:16:03.847510 26893 leader_election.cc:288] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127), 25e23785489c43d0a39e77a995650c42 (127.18.166.253:37805)
I20260812 05:16:03.847919 26894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.847672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45516)
0812 05:16:03.847800 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.847874 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
W20260812 05:16:03.848073 26333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.848469 26894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.848353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45528)
0812 05:16:03.848412 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.848443 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 05:16:03.848577 26333 leader_election.cc:334] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127): Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.850690 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.847726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:37805 (local address 127.0.0.1:44746)
0812 05:16:03.847839 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.847851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:03.847920 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:03.848116 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:03.848117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:03.848119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:03.848199 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.848202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.849013 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.849014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.849873 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.849875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.850429 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.850430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.850499 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.850502 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:03.850579 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 05:16:03.850622 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260812 05:16:03.850696 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.847790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.18.166.253:37805)
0812 05:16:03.847921 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:03.847924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:03.847961 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:16:03.847994 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:03.847995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:03.848007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:03.848066 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:03.848277 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.848927 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.849974 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.850347 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.850433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.850442 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:03.850587 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 05:16:03.850629 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260812 05:16:03.851011 26154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de028c7533a3448da2355a31573994b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "25e23785489c43d0a39e77a995650c42" is_pre_election: true
I20260812 05:16:03.851300 26330 leader_election.cc:302] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de028c7533a3448da2355a31573994b0; no voters: 25e23785489c43d0a39e77a995650c42, 4cbc350ebb29477ba381a76b5abd0ed7
I20260812 05:16:03.851415 26893 raft_consensus.cc:2726] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:16:03.854890 26898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4cbc350ebb29477ba381a76b5abd0ed7)
I20260812 05:16:03.854970 26898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } }
I20260812 05:16:03.855239 26898 leader_election.cc:288] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127), de028c7533a3448da2355a31573994b0 (127.18.166.252:37123)
I20260812 05:16:03.855537 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.855348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45552)
0812 05:16:03.855456 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.855509 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
W20260812 05:16:03.855665 26131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.856127 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.856038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45558)
0812 05:16:03.856068 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.856093 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 05:16:03.856215 26131 leader_election.cc:334] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127): Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.858124 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.855369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:37123 (local address 127.0.0.1:34612)
0812 05:16:03.855614 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.855624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:03.855675 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:03.855873 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:03.855873 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:03.855876 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:03.855923 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.855924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.856639 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.856640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.857377 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.857379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.857956 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.857957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.858004 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.858007 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:03.858050 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 05:16:03.858080 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":41,"threads_started":1}
I20260812 05:16:03.858263 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.855495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.18.166.252:37123)
0812 05:16:03.855709 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:03.855717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:03.855725 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:03.855764 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:03.855764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:03.855777 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:03.855868 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:03.856009 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.856564 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.857525 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.857880 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.857937 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.857943 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:03.858093 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 05:16:03.858136 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260812 05:16:03.858444 26356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25e23785489c43d0a39e77a995650c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de028c7533a3448da2355a31573994b0" is_pre_election: true
I20260812 05:16:03.858551 26356 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25e23785489c43d0a39e77a995650c42 in term 1.
I20260812 05:16:03.858707 26129 leader_election.cc:302] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25e23785489c43d0a39e77a995650c42, de028c7533a3448da2355a31573994b0; no voters: 4cbc350ebb29477ba381a76b5abd0ed7
I20260812 05:16:03.858812 26898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 05:16:03.858852 26898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4cbc350ebb29477ba381a76b5abd0ed7)
I20260812 05:16:03.858886 26898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:16:03.859668 26898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } }
I20260812 05:16:03.859869 26898 leader_election.cc:288] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [CANDIDATE]: Term 2 election: Requested vote from peers 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127), de028c7533a3448da2355a31573994b0 (127.18.166.252:37123)
I20260812 05:16:03.860081 26356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "25e23785489c43d0a39e77a995650c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de028c7533a3448da2355a31573994b0"
I20260812 05:16:03.860117 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.859944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45564)
0812 05:16:03.860064 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.860094 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 05:16:03.860152 26356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:16:03.860852 26356 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25e23785489c43d0a39e77a995650c42 in term 2.
I20260812 05:16:03.861069 26129 leader_election.cc:302] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e23785489c43d0a39e77a995650c42, de028c7533a3448da2355a31573994b0; no voters:
I20260812 05:16:03.861186 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.861083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45568)
0812 05:16:03.861132 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.861157 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:16:03.861202 26898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 2 FOLLOWER]: Leader election won for term 2
W20260812 05:16:03.861333 26131 leader_election.cc:334] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127): Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
I20260812 05:16:03.861356 26898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [term 2 LEADER]: Becoming Leader. State: Replica: 25e23785489c43d0a39e77a995650c42, State: Running, Role: LEADER
I20260812 05:16:03.861483 26898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } }
I20260812 05:16:03.861876 26903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e23785489c43d0a39e77a995650c42. Latest consensus state: current_term: 2 leader_uuid: "25e23785489c43d0a39e77a995650c42" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:03.861981 26903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:03.862169 26905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:03.862656 26905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:16:03.862771 26905 catalog_manager.cc:1259] Loaded cluster ID: 116db59afc0442879118835f9a646747
I20260812 05:16:03.862805 26905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:03.862979 26905 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:03.863075 26905 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Loaded TSK: 0
I20260812 05:16:03.863214 26905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:16:03.914204 26143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:16:03.915834 26143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:03.916937 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.916821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45586)
0812 05:16:03.916877 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.916912 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:16:03.916956 26356 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [term 2 FOLLOWER]: Refusing update from remote peer 25e23785489c43d0a39e77a995650c42: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 05:16:03.917302 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.917164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.0.0.1:45602)
0812 05:16:03.917193 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.917235 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 05:16:03.917248 26903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 05:16:03.917393 26131 consensus_peers.cc:487] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 -> Peer 4cbc350ebb29477ba381a76b5abd0ed7 (127.18.166.254:35127): Couldn't send request to peer 4cbc350ebb29477ba381a76b5abd0ed7. Status: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 05:16:03.917896 26893 sys_catalog.cc:455] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e23785489c43d0a39e77a995650c42. Latest consensus state: current_term: 2 leader_uuid: "25e23785489c43d0a39e77a995650c42" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:03.918015 26893 sys_catalog.cc:458] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:16:03.918354 26906 mvcc.cc:205] Tried to move back new op lower bound from 7317552185001631744 to 7317552184777424896. Current Snapshot: MvccSnapshot[applied={T|T < 7317552185001631744}]
I20260812 05:16:03.918447 26898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "25e23785489c43d0a39e77a995650c42" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:03.918543 26898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:03.918503 26907 mvcc.cc:205] Tried to move back new op lower bound from 7317552185001631744 to 7317552184777424896. Current Snapshot: MvccSnapshot[applied={T|T < 7317552185001631744}]
I20260812 05:16:03.918756 26893 sys_catalog.cc:455] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "25e23785489c43d0a39e77a995650c42" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cbc350ebb29477ba381a76b5abd0ed7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35127 } } peers { permanent_uuid: "25e23785489c43d0a39e77a995650c42" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 37805 } attrs { promote: false } } peers { permanent_uuid: "de028c7533a3448da2355a31573994b0" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 37123 } attrs { promote: false } } }
I20260812 05:16:03.918857 26893 sys_catalog.cc:458] T 00000000000000000000000000000000 P de028c7533a3448da2355a31573994b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:16:03.924132 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.920452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:36853 (local address 127.0.0.1:37232)
0812 05:16:03.920529 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.920540 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:03.920581 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:03.921305 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:03.921306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:03.921309 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:03.921340 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.921341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.922253 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.922254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.923075 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.923078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.923849 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.923912 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.923917 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:16:03.923975 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:16:03.924025 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":3}
I20260812 05:16:03.924256 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.920526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:46791 (local address 127.0.0.1:39936)
0812 05:16:03.920623 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.920635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:03.920686 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:03.921037 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:03.921038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:03.921041 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:03.921089 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.921091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.922064 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.922064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.923243 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.923245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.923979 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.924068 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.924077 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 05:16:03.924144 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:16:03.924186 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":42}
I20260812 05:16:03.924273 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.920709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.18.166.195:46791)
0812 05:16:03.920894 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:03.920897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:03.920905 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:03.920931 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:03.920932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:03.920942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:03.921032 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:03.921166 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.921986 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923324 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.923901 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923974 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.923983 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:03.924149 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 05:16:03.924196 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260812 05:16:03.924403 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.920531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38983 (local address 127.0.0.1:36678)
0812 05:16:03.921009 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.921018 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:03.921067 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:03.921266 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:03.921267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:03.921269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:03.921318 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.921320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.921987 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.921988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.923449 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:03.923450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.924161 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.924162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:03.924238 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.924242 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:03.924291 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 05:16:03.924331 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":300,"thread_start_us":88,"threads_started":1}
I20260812 05:16:03.924641 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.920634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.18.166.194:38983)
0812 05:16:03.920875 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:03.920877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:03.921108 (+ 231us) server_negotiation.cc:407] Connection header received
0812 05:16:03.921143 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:03.921143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:03.921155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:03.921215 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:03.921374 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.921891 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923541 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.924038 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.924118 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.924127 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:03.924527 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 05:16:03.924572 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
I20260812 05:16:03.924641 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.920535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37232 (local address 127.18.166.193:36853)
0812 05:16:03.921163 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:03.921165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:03.921185 (+ 20us) server_negotiation.cc:407] Connection header received
0812 05:16:03.921212 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:03.921213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:03.921223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:03.921286 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:03.921461 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.922171 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923179 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:03.923763 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:03.923843 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:03.923852 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:03.924283 (+ 431us) server_negotiation.cc:299] Negotiation successful
0812 05:16:03.924334 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":70,"threads_started":1}
I20260812 05:16:03.924762 26755 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffc46361c7c46a0bfe27a122dadb6c8 (DEFAULT_TABLE table=test-table [id=790d3e599ea349f3a6fbcb66d57bd36e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:03.924865 26468 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffc46361c7c46a0bfe27a122dadb6c8 (DEFAULT_TABLE table=test-table [id=790d3e599ea349f3a6fbcb66d57bd36e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:03.924881 26614 tablet_service.cc:1467] Processing CreateTablet for tablet 1ffc46361c7c46a0bfe27a122dadb6c8 (DEFAULT_TABLE table=test-table [id=790d3e599ea349f3a6fbcb66d57bd36e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:03.924917 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffc46361c7c46a0bfe27a122dadb6c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:03.924993 26614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffc46361c7c46a0bfe27a122dadb6c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:03.924993 26468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ffc46361c7c46a0bfe27a122dadb6c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:03.926638 26916 tablet_bootstrap.cc:492] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Bootstrap starting.
I20260812 05:16:03.926666 26918 tablet_bootstrap.cc:492] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Bootstrap starting.
I20260812 05:16:03.927085 26918 tablet_bootstrap.cc:654] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:03.927086 26916 tablet_bootstrap.cc:654] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:03.927415 26917 tablet_bootstrap.cc:492] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d: Bootstrap starting.
I20260812 05:16:03.927767 26918 tablet_bootstrap.cc:492] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: No bootstrap required, opened a new log
I20260812 05:16:03.927809 26917 tablet_bootstrap.cc:654] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:03.927829 26918 ts_tablet_manager.cc:1397] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:03.927968 26916 tablet_bootstrap.cc:492] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: No bootstrap required, opened a new log
I20260812 05:16:03.927954 26918 raft_consensus.cc:348] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:03.928030 26916 ts_tablet_manager.cc:1397] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:16:03.928048 26918 raft_consensus.cc:374] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:03.928085 26918 raft_consensus.cc:729] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00365876de384b3b8745214b55861708, State: Initialized, Role: FOLLOWER
I20260812 05:16:03.928140 26918 consensus_queue.cc:260] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:03.928169 26916 raft_consensus.cc:348] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:03.928246 26916 raft_consensus.cc:374] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:03.928278 26916 raft_consensus.cc:729] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72cb9c007d54c6ca0150247f1251ea0, State: Initialized, Role: FOLLOWER
I20260812 05:16:03.928294 26918 ts_tablet_manager.cc:1428] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:03.928527 26916 consensus_queue.cc:260] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:03.928758 26916 ts_tablet_manager.cc:1428] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:03.928939 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.928550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:44933)
0812 05:16:03.928853 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.928895 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":85,"threads_started":1}
I20260812 05:16:03.929205 26684 heartbeater.cc:503] Master 127.18.166.253:37805 was elected leader, sending a full tablet report...
I20260812 05:16:03.929330 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.929155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:42807)
0812 05:16:03.929272 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.929300 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260812 05:16:03.929466 26824 heartbeater.cc:503] Master 127.18.166.253:37805 was elected leader, sending a full tablet report...
I20260812 05:16:03.929809 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.928947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:43913)
0812 05:16:03.929728 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.929767 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":74,"threads_started":1}
I20260812 05:16:03.930141 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.930035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:38725)
0812 05:16:03.930092 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.930117 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:16:03.930315 26917 tablet_bootstrap.cc:492] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d: No bootstrap required, opened a new log
I20260812 05:16:03.930377 26917 ts_tablet_manager.cc:1397] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:16:03.930514 26917 raft_consensus.cc:348] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:03.930598 26917 raft_consensus.cc:374] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:03.930621 26917 raft_consensus.cc:729] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5989a69309634f8b910185d08e18992d, State: Initialized, Role: FOLLOWER
I20260812 05:16:03.930670 26917 consensus_queue.cc:260] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:03.930845 26917 ts_tablet_manager.cc:1428] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:03.931233 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.930988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:40197)
0812 05:16:03.931131 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.931183 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260812 05:16:03.931277 26541 heartbeater.cc:503] Master 127.18.166.253:37805 was elected leader, sending a full tablet report...
I20260812 05:16:03.931596 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:03.931430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:37117)
0812 05:16:03.931538 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:03.931564 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260812 05:16:04.123821 26924 raft_consensus.cc:482] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:04.123929 26924 raft_consensus.cc:504] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:04.124338 26924 leader_election.cc:288] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5989a69309634f8b910185d08e18992d (127.18.166.193:36853), d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:04.127667 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.124565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:46791 (local address 127.18.166.194:46123)
0812 05:16:04.124643 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.124659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:04.124751 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:04.124916 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:04.124917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:04.124919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:04.125009 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:04.125011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.125832 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.125834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:04.126773 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:04.126776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.127435 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.127436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:04.127504 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:04.127508 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:04.127570 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:16:04.127602 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:16:04.127719 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.124730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:46123 (local address 127.18.166.195:46791)
0812 05:16:04.124784 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:04.124786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:04.124795 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:04.124826 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:04.124827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:04.124836 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:04.124913 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:04.125097 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.125726 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.126944 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.127353 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.127410 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:04.127418 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:04.127585 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 05:16:04.127623 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:16:04.127839 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.124664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:53181 (local address 127.18.166.193:36853)
0812 05:16:04.124712 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:04.124714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:04.124956 (+ 242us) server_negotiation.cc:407] Connection header received
0812 05:16:04.124987 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:04.124988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:04.124998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:04.125056 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:04.125245 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.125901 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.127001 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.127439 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.127510 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:04.127521 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:04.127724 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 05:16:04.127771 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:16:04.127909 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.124565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:36853 (local address 127.18.166.194:53181)
0812 05:16:04.124693 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.124706 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:04.124770 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:04.125083 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:04.125085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:04.125086 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:04.125159 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:04.125160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.126005 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.126007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:04.126901 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:04.126903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.127537 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.127555 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:04.127636 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:04.127639 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:04.127706 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:16:04.127751 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":73}
I20260812 05:16:04.127954 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffc46361c7c46a0bfe27a122dadb6c8" candidate_uuid: "00365876de384b3b8745214b55861708" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72cb9c007d54c6ca0150247f1251ea0" is_pre_election: true
I20260812 05:16:04.128059 26775 raft_consensus.cc:2454] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00365876de384b3b8745214b55861708 in term 0.
I20260812 05:16:04.128156 26488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffc46361c7c46a0bfe27a122dadb6c8" candidate_uuid: "00365876de384b3b8745214b55861708" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989a69309634f8b910185d08e18992d" is_pre_election: true
I20260812 05:16:04.128296 26488 raft_consensus.cc:2454] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00365876de384b3b8745214b55861708 in term 0.
I20260812 05:16:04.128487 26566 leader_election.cc:302] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00365876de384b3b8745214b55861708, 5989a69309634f8b910185d08e18992d; no voters:
I20260812 05:16:04.128630 26924 raft_consensus.cc:2781] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:04.128679 26924 raft_consensus.cc:482] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:04.128705 26924 raft_consensus.cc:3037] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:04.129346 26924 raft_consensus.cc:504] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:04.129541 26924 leader_election.cc:288] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [CANDIDATE]: Term 1 election: Requested vote from peers 5989a69309634f8b910185d08e18992d (127.18.166.193:36853), d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791)
I20260812 05:16:04.129703 26488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffc46361c7c46a0bfe27a122dadb6c8" candidate_uuid: "00365876de384b3b8745214b55861708" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5989a69309634f8b910185d08e18992d"
I20260812 05:16:04.129724 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ffc46361c7c46a0bfe27a122dadb6c8" candidate_uuid: "00365876de384b3b8745214b55861708" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72cb9c007d54c6ca0150247f1251ea0"
I20260812 05:16:04.129781 26488 raft_consensus.cc:3037] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:04.129791 26775 raft_consensus.cc:3037] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:04.130537 26775 raft_consensus.cc:2454] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00365876de384b3b8745214b55861708 in term 1.
I20260812 05:16:04.130707 26488 raft_consensus.cc:2454] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00365876de384b3b8745214b55861708 in term 1.
I20260812 05:16:04.130710 26565 leader_election.cc:302] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00365876de384b3b8745214b55861708, d72cb9c007d54c6ca0150247f1251ea0; no voters:
I20260812 05:16:04.130823 26924 raft_consensus.cc:2781] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:04.130981 26924 raft_consensus.cc:686] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 1 LEADER]: Becoming Leader. State: Replica: 00365876de384b3b8745214b55861708, State: Running, Role: LEADER
I20260812 05:16:04.131150 26924 consensus_queue.cc:237] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } }
I20260812 05:16:04.131348 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.131233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:58383)
0812 05:16:04.131284 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.131320 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 05:16:04.131969 26143 catalog_manager.cc:5495] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00365876de384b3b8745214b55861708 (127.18.166.194). New cstate: current_term: 1 leader_uuid: "00365876de384b3b8745214b55861708" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00365876de384b3b8745214b55861708" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38983 } health_report { overall_health: HEALTHY } } }
I20260812 05:16:04.150208 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.148016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.18.166.194:38983)
0812 05:16:04.148063 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:04.148070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:04.148078 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:04.148145 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:04.148146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:04.148164 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:04.148231 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:04.148360 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.148828 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.149576 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:04.149762 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:04.149821 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:04.149932 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:04.150028 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:04.150075 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:04.150123 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 05:16:04.150140 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:16:04.151376 26775 raft_consensus.cc:1261] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Refusing update from remote peer 00365876de384b3b8745214b55861708: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:04.151499 26488 raft_consensus.cc:1261] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 5989a69309634f8b910185d08e18992d [term 1 FOLLOWER]: Refusing update from remote peer 00365876de384b3b8745214b55861708: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:04.151669 26932 consensus_queue.cc:1035] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72cb9c007d54c6ca0150247f1251ea0" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:04.151808 26924 consensus_queue.cc:1035] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5989a69309634f8b910185d08e18992d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:04.152560 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.152443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:46265)
0812 05:16:04.152481 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.152526 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 05:16:04.152807 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.152681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:42473)
0812 05:16:04.152731 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.152765 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:16:04.154117 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.153904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:49273)
0812 05:16:04.154052 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.154085 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260812 05:16:04.154567 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.154462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.193:52107)
0812 05:16:04.154504 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.154538 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 05:16:04.154958 26935 mvcc.cc:205] Tried to move back new op lower bound from 7317552185961943040 to 7317552185881874432. Current Snapshot: MvccSnapshot[applied={T|T < 7317552185961943040 or (T in {7317552185961943040})}]
I20260812 05:16:04.156162 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.155970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.194:39075)
0812 05:16:04.156108 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.156135 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 05:16:04.156875 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:04.156780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35127 (local address 127.18.166.195:37821)
0812 05:16:04.156818 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.156844 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:35127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 05:16:04.157269 26144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:04.157373 26144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:04.158808 26144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25e23785489c43d0a39e77a995650c42: Sending DeleteTablet for 3 replicas of tablet 1ffc46361c7c46a0bfe27a122dadb6c8
I20260812 05:16:04.159101 26755 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ffc46361c7c46a0bfe27a122dadb6c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:04 UTC) from {username='oryx', principal='oryx/127.18.166.253@KRBTEST.COM'} at 127.0.0.1:39936
I20260812 05:16:04.159292 26468 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ffc46361c7c46a0bfe27a122dadb6c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:04 UTC) from {username='oryx', principal='oryx/127.18.166.253@KRBTEST.COM'} at 127.0.0.1:37232
I20260812 05:16:04.159307 26614 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ffc46361c7c46a0bfe27a122dadb6c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:04 UTC) from {username='oryx', principal='oryx/127.18.166.253@KRBTEST.COM'} at 127.0.0.1:36678
I20260812 05:16:04.159490 26948 tablet_replica.cc:330] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: stopping tablet replica
I20260812 05:16:04.159614 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 26308
I20260812 05:16:04.159554 26948 raft_consensus.cc:2229] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:04.164332 26948 raft_consensus.cc:2258] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:04.164726 26948 ts_tablet_manager.cc:1905] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:04.166473 26948 ts_tablet_manager.cc:1918] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:04.166560 26948 log.cc:1196] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/1ffc46361c7c46a0bfe27a122dadb6c8
I20260812 05:16:04.166842 26948 ts_tablet_manager.cc:1939] T 1ffc46361c7c46a0bfe27a122dadb6c8 P 00365876de384b3b8745214b55861708: Deleting consensus metadata
I20260812 05:16:04.167589 26130 catalog_manager.cc:4841] TS 00365876de384b3b8745214b55861708 (127.18.166.194:38983): tablet 1ffc46361c7c46a0bfe27a122dadb6c8 (table test-table [id=790d3e599ea349f3a6fbcb66d57bd36e]) successfully deleted
I20260812 05:16:04.169545 26946 tablet_replica.cc:330] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: stopping tablet replica
I20260812 05:16:04.169632 26946 raft_consensus.cc:2229] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:04.169692 26946 raft_consensus.cc:2258] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:04.170047 26946 ts_tablet_manager.cc:1905] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 05:16:04.171000 26915 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:04.170873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:36853 (local address 127.0.0.1:37246)
0812 05:16:04.170927 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:04.170954 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:36853: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:16:04.171339 26129 catalog_manager.cc:4584] TS 5989a69309634f8b910185d08e18992d (127.18.166.193:36853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ffc46361c7c46a0bfe27a122dadb6c8: Network error: Client connection negotiation failed: client connection to 127.18.166.193:36853: connect: Connection reset by peer (error 104)
I20260812 05:16:04.171527 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 26548
I20260812 05:16:04.171814 26946 ts_tablet_manager.cc:1918] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:04.171869 26946 log.cc:1196] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/1ffc46361c7c46a0bfe27a122dadb6c8
I20260812 05:16:04.172573 26946 ts_tablet_manager.cc:1939] T 1ffc46361c7c46a0bfe27a122dadb6c8 P d72cb9c007d54c6ca0150247f1251ea0: Deleting consensus metadata
I20260812 05:16:04.172976 26128 catalog_manager.cc:4841] TS d72cb9c007d54c6ca0150247f1251ea0 (127.18.166.195:46791): tablet 1ffc46361c7c46a0bfe27a122dadb6c8 (table test-table [id=790d3e599ea349f3a6fbcb66d57bd36e]) successfully deleted
I20260812 05:16:04.178565 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 26689
I20260812 05:16:04.184310 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 26001
I20260812 05:16:04.190672 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 26199
2026-08-12T05:16:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5133 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:16:04 dist-test-slave-srgv krb5kdc[26951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:16:04 dist-test-slave-srgv krb5kdc[26951](info): set up 2 sockets
Aug 12 05:16:04 dist-test-slave-srgv krb5kdc[26951](info): commencing operation
krb5kdc: starting...
W20260812 05:16:05.263737 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:16:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab.
Entry for principal oryx/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab.
I20260812 05:16:05.387928 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:37715
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab
--principal=oryx/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:37715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:05.470140 26965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:05.470394 26965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:05.470432 26965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:05.471606 26965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:05.471673 26965 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 05:16:05.471696 26965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:05.471719 26965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:05.471748 26965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:05.471773 26965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:05.473311 26965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:37715
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.254.keytab
--principal=oryx/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:37715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:05.473634 26965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:05.473869 26965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:05.478919 26971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:05.479002 26965 server_base.cc:1034] running on GCE node
W20260812 05:16:05.478912 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:05.478912 26970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:05.479354 26965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:05.479650 26965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:05.480860 26965 hybrid_clock.cc:648] HybridClock initialized: now 1786511765480847 us; error 26 us; skew 500 ppm
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:05.483395 26965 init.cc:377] Logged in from keytab as oryx/127.18.166.254@KRBTEST.COM (short username oryx)
I20260812 05:16:05.483807 26965 webserver.cc:460] Webserver started at http://127.18.166.254:44665/ using document root <none> and password file <none>
I20260812 05:16:05.484011 26965 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:05.484069 26965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:05.484160 26965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:05.485179 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "0440ae18699a40f58120dcb862bef859"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "dfc832e0c1569397804a6076687e6a71"
server_key_iv: "534f82f766337b18a7352f974e38518c"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.485562 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "0440ae18699a40f58120dcb862bef859"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "dfc832e0c1569397804a6076687e6a71"
server_key_iv: "534f82f766337b18a7352f974e38518c"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.486878 26965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:05.487625 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:05.487845 26965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:05.487928 26965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "0440ae18699a40f58120dcb862bef859"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "dfc832e0c1569397804a6076687e6a71"
server_key_iv: "534f82f766337b18a7352f974e38518c"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.487985 26965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:05.504689 26965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:05.507262 26965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:05.507385 26965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:05.512156 26965 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:37715
I20260812 05:16:05.512267 27043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:37715 every 8 connection(s)
I20260812 05:16:05.512612 26965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:05.513191 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 26965
I20260812 05:16:05.513271 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:16:05.513499 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:05.513548 19099 external_mini_cluster.cc:1428] Setting key f5e218caeb7cb9bdaa604a5c4254405b
I20260812 05:16:05.517226 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Bootstrap starting.
I20260812 05:16:05.517870 27044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:05.518187 27044 log.cc:824] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:05.518908 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: No bootstrap required, opened a new log
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:05.521068 27044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0440ae18699a40f58120dcb862bef859" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 37715 } }
I20260812 05:16:05.521202 27044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:05.521234 27044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0440ae18699a40f58120dcb862bef859, State: Initialized, Role: FOLLOWER
I20260812 05:16:05.521389 27044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0440ae18699a40f58120dcb862bef859" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 37715 } }
I20260812 05:16:05.521512 27044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:05.521536 27044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:05.521610 27044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:05.522140 27044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0440ae18699a40f58120dcb862bef859" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 37715 } }
I20260812 05:16:05.522255 27044 leader_election.cc:302] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0440ae18699a40f58120dcb862bef859; no voters:
I20260812 05:16:05.522472 27044 leader_election.cc:288] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:16:05.522505 27050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:05.522610 27050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [term 1 LEADER]: Becoming Leader. State: Replica: 0440ae18699a40f58120dcb862bef859, State: Running, Role: LEADER
I20260812 05:16:05.522681 27050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0440ae18699a40f58120dcb862bef859" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 37715 } }
I20260812 05:16:05.522749 27044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:05.523007 27050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0440ae18699a40f58120dcb862bef859. Latest consensus state: current_term: 1 leader_uuid: "0440ae18699a40f58120dcb862bef859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0440ae18699a40f58120dcb862bef859" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 37715 } } }
I20260812 05:16:05.523079 27050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:05.524506 27052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0440ae18699a40f58120dcb862bef859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0440ae18699a40f58120dcb862bef859" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 37715 } } }
I20260812 05:16:05.524612 27052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859 [sys.catalog]: This master's current role is: LEADER
W20260812 05:16:05.524762 27067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:16:05.524834 27067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:16:05.524893 27056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:05.525082 27056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:16:05.525573 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.514085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.18.166.254:37715)
0812 05:16:05.514285 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:05.514294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:05.514327 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:16:05.514985 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:05.514993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:05.515055 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:05.515211 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:05.515854 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.517251 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.518141 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.518643 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.518768 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.521478 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:05.521483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:05.521484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:05.521519 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:05.523804 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.524201 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.524205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.524205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.524313 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.524564 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.524565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.524565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.524843 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:05.525018 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:05.525166 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 05:16:05.525333 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":120,"threads_started":1}
I20260812 05:16:05.526664 27056 catalog_manager.cc:1347] Generated new cluster ID: 7c069e90c83541959cb27d9c301e30e4
I20260812 05:16:05.526719 27056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:05.552263 27056 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:16:05.553025 27056 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:05.567610 27056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Generated new TSK 0
I20260812 05:16:05.567847 27056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab.
Entry for principal oryx/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab.
I20260812 05:16:05.596424 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:37715
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:05.679702 27072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:05.679900 27072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:05.679924 27072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:05.681085 27072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:05.681128 27072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:05.681149 27072 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:16:05.681195 27072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:05.682740 27072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:37715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:05.682982 27072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:05.683228 27072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:05.688760 27082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:05.688783 27079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:05.688760 27078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:05.688787 27072 server_base.cc:1034] running on GCE node
I20260812 05:16:05.689145 27072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:05.689350 27072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:05.690493 27072 hybrid_clock.cc:648] HybridClock initialized: now 1786511765690481 us; error 29 us; skew 500 ppm
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:05.693148 27072 init.cc:377] Logged in from keytab as oryx/127.18.166.193@KRBTEST.COM (short username oryx)
I20260812 05:16:05.693638 27072 webserver.cc:460] Webserver started at http://127.18.166.193:39389/ using document root <none> and password file <none>
I20260812 05:16:05.693881 27072 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:05.693950 27072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:05.694115 27072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:05.695156 27072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "fdb9a9ab2c88ee1d1297a53297890f20"
server_key_iv: "ec917005a6c02e346031e68d645b7c4b"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.695519 27072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "fdb9a9ab2c88ee1d1297a53297890f20"
server_key_iv: "ec917005a6c02e346031e68d645b7c4b"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.696846 27072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:16:05.697614 27089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:05.697805 27072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:05.697896 27072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "fdb9a9ab2c88ee1d1297a53297890f20"
server_key_iv: "ec917005a6c02e346031e68d645b7c4b"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.697983 27072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:05.706893 27072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:05.709177 27072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:05.709339 27072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:05.709870 27072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:05.709935 27072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:05.709981 27072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:05.710006 27072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:05.717136 27072 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:33963
I20260812 05:16:05.717228 27213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:33963 every 8 connection(s)
I20260812 05:16:05.717573 27072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:05.720080 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.710350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:48213 (local address 127.18.166.254:37715)
0812 05:16:05.710395 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:05.710397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:05.710616 (+ 219us) server_negotiation.cc:407] Connection header received
0812 05:16:05.711153 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:05.711154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:05.711168 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:05.711230 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:05.712074 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.712883 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.714623 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.714873 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.714955 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.717369 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:05.717371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:05.717371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:05.717380 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:05.718924 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.719297 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.719298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.719298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.719349 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.719519 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.719519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.719519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.719618 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:05.719670 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:05.719948 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 05:16:05.720016 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:16:05.720345 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.710341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.18.166.193:48213)
0812 05:16:05.710493 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:05.710527 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:05.711036 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:05.711300 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:05.711304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:05.711687 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:05.711971 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.711977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.713010 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.713011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.714524 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.714528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.714966 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.714967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.715098 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.715641 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:05.715651 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:05.717102 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:05.719003 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.719008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.719009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.719230 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.719389 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.719389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.719389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.719470 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.719701 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:05.719708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:05.719827 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 05:16:05.719927 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 05:16:05.720052 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260812 05:16:05.720890 27214 heartbeater.cc:347] Connected to a master server at 127.18.166.254:37715
I20260812 05:16:05.720970 27214 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:05.721145 27214 heartbeater.cc:511] Master 127.18.166.254:37715 requested a full tablet report, sending...
I20260812 05:16:05.721707 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 27072
I20260812 05:16:05.721763 26996 ts_manager.cc:194] Registered new tserver with Master: fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963)
I20260812 05:16:05.721822 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:16:05.722045 19099 external_mini_cluster.cc:1428] Setting key d793838106a2c43738bd8f18bda3250a
I20260812 05:16:05.722681 26996 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:48213
I20260812 05:16:05.728220 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.722317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:40265 (local address 127.18.166.254:37715)
0812 05:16:05.722380 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:05.722383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:05.722390 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:16:05.722407 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:05.722407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:05.722416 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:05.722491 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:05.722947 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.723541 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.724445 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.724702 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.724778 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.725766 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:05.725767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:05.725767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:05.725774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:05.727097 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.727439 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.727440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.727441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.727492 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.727705 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.727706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.727707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.727820 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:05.727893 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:05.728053 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 05:16:05.728127 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:16:05.728241 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.722025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.18.166.193:40265)
0812 05:16:05.722172 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:05.722183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:05.722233 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:05.722499 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:05.722500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:05.722723 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:05.722796 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.722800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.723637 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.723638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.724367 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.724370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.724778 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.724779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.724847 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.725311 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:05.725316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:05.725668 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:05.727182 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.727183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.727183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.727367 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.727571 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.727571 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.727572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.727635 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.727897 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:05.727898 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:05.727943 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 05:16:05.728027 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:16:05.728130 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab.
Entry for principal oryx/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab.
I20260812 05:16:05.749176 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:37715
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:05.832064 27224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:05.832288 27224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:05.832314 27224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:05.833437 27224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:05.833474 27224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:05.833495 27224 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:16:05.833513 27224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:05.835055 27224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:37715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:05.835246 27224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:05.835427 27224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:05.840482 27230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:05.840435 27229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:05.840431 27232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:05.840587 27224 server_base.cc:1034] running on GCE node
I20260812 05:16:05.840962 27224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:05.841171 27224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:05.842355 27224 hybrid_clock.cc:648] HybridClock initialized: now 1786511765842346 us; error 27 us; skew 500 ppm
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:05.844990 27224 init.cc:377] Logged in from keytab as oryx/127.18.166.194@KRBTEST.COM (short username oryx)
I20260812 05:16:05.845531 27224 webserver.cc:460] Webserver started at http://127.18.166.194:42073/ using document root <none> and password file <none>
I20260812 05:16:05.845778 27224 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:05.845852 27224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:05.845990 27224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:05.847046 27224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "6546b343d096402abf1916ec60b7aa6b"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "7c2b7e807131d43f55ac10cc8ada9f26"
server_key_iv: "c868fc2fc3fefa8da3c16df2d387e7c2"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.847433 27224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "6546b343d096402abf1916ec60b7aa6b"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "7c2b7e807131d43f55ac10cc8ada9f26"
server_key_iv: "c868fc2fc3fefa8da3c16df2d387e7c2"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.848793 27224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:16:05.849550 27239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:05.849726 27224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:05.849838 27224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "6546b343d096402abf1916ec60b7aa6b"
format_stamp: "Formatted at 2026-08-12 05:16:05 on dist-test-slave-srgv"
server_key: "7c2b7e807131d43f55ac10cc8ada9f26"
server_key_iv: "c868fc2fc3fefa8da3c16df2d387e7c2"
server_key_version: "encryptionkey@0"
I20260812 05:16:05.849925 27224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:05.874650 27224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:05.877141 27224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:05.877326 27224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:05.877894 27224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:05.877961 27224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:05.878003 27224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:05.878032 27224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:05.885236 27224 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:46739
I20260812 05:16:05.885329 27358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:46739 every 8 connection(s)
Aug 12 05:16:05 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:05.885713 27224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:05.888501 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.878376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:33015 (local address 127.18.166.254:37715)
0812 05:16:05.878419 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:05.878421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:05.878659 (+ 238us) server_negotiation.cc:407] Connection header received
0812 05:16:05.879239 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:05.879240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:05.879254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:05.879304 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:05.880184 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.881004 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.882741 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.882995 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.883090 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.886128 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:05.886131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:05.886132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:05.886140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:05.887338 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.887703 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.887704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.887704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.887756 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.887982 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.887983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.887983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.888089 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:05.888139 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:05.888367 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 05:16:05.888437 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:16:05.888654 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.878374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.18.166.194:33015)
0812 05:16:05.878535 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:05.878571 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:05.879100 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:05.879382 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:05.879385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:05.879776 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:05.880066 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.880092 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.881122 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.881123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.882642 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.882646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.883096 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.883097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.883249 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.883973 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:05.883984 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:05.886003 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:05.887406 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.887413 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.887414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.887634 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.887828 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.887829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.887829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.887926 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.888142 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:05.888146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:05.888258 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 05:16:05.888339 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 05:16:05.888460 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260812 05:16:05.889133 27359 heartbeater.cc:347] Connected to a master server at 127.18.166.254:37715
I20260812 05:16:05.889223 27359 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:05.889438 27359 heartbeater.cc:511] Master 127.18.166.254:37715 requested a full tablet report, sending...
I20260812 05:16:05.889845 26997 ts_manager.cc:194] Registered new tserver with Master: 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739)
I20260812 05:16:05.890359 26997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:33015
I20260812 05:16:05.894786 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 27224
I20260812 05:16:05.894907 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:16:05.895162 19099 external_mini_cluster.cc:1428] Setting key 560154aa5b1bfe157f863ae6a0f0b50c
I20260812 05:16:05.895846 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.890271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:49771 (local address 127.18.166.254:37715)
0812 05:16:05.890343 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:05.890345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:05.890465 (+ 120us) server_negotiation.cc:407] Connection header received
0812 05:16:05.890517 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:05.890518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:05.890529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:05.890592 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:05.890988 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.891514 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.892187 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.892362 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.892409 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.893309 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:05.893309 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:05.893310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:05.893314 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:05.894499 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.894906 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.894907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.894908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.894988 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:05.895232 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:05.895232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:05.895233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:05.895449 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:05.895516 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:05.895674 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 05:16:05.895735 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 05:16:05.895908 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:05.890217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.18.166.194:49771)
0812 05:16:05.890382 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:05.890394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:05.890464 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:05.890596 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:05.890597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:05.890828 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:05.890894 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.890896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.891619 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.891620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.892120 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:05.892121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:05.892425 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:05.892426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:05.892472 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:05.892943 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:05.892948 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:05.893237 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:05.894589 (+ 1352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.894590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.894591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.894821 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.895076 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:05.895076 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:05.895077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:05.895147 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:05.895529 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:05.895530 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:05.895576 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 05:16:05.895655 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 05:16:05.895772 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab.
Entry for principal oryx/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab.
I20260812 05:16:05.922899 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:37715
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:06.008888 27368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:06.009093 27368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:06.009117 27368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:06.010231 27368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:06.010267 27368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:06.010286 27368 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:16:06.010303 27368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:06.011770 27368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.166.254:37715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:06.011956 27368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:06.012136 27368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:16:06.017200 27368 server_base.cc:1034] running on GCE node
W20260812 05:16:06.017242 27374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:06.017246 27373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:06.017225 27376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:06.017624 27368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:06.017897 27368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:06.019068 27368 hybrid_clock.cc:648] HybridClock initialized: now 1786511766019055 us; error 31 us; skew 500 ppm
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511766, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:06.021598 27368 init.cc:377] Logged in from keytab as oryx/127.18.166.195@KRBTEST.COM (short username oryx)
I20260812 05:16:06.022130 27368 webserver.cc:460] Webserver started at http://127.18.166.195:41125/ using document root <none> and password file <none>
I20260812 05:16:06.022414 27368 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:06.022467 27368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:06.022588 27368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:06.023874 27368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "a06500afa15f4806b343abb20d1021fe"
format_stamp: "Formatted at 2026-08-12 05:16:06 on dist-test-slave-srgv"
server_key: "23bc6b45fe101e5f57f3de07b0fe13bf"
server_key_iv: "6e5a0f69e21f443309a5d763a5b44195"
server_key_version: "encryptionkey@0"
I20260812 05:16:06.024284 27368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "a06500afa15f4806b343abb20d1021fe"
format_stamp: "Formatted at 2026-08-12 05:16:06 on dist-test-slave-srgv"
server_key: "23bc6b45fe101e5f57f3de07b0fe13bf"
server_key_iv: "6e5a0f69e21f443309a5d763a5b44195"
server_key_version: "encryptionkey@0"
I20260812 05:16:06.025844 27368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 05:16:06.026597 27383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.026803 27368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:06.026894 27368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "a06500afa15f4806b343abb20d1021fe"
format_stamp: "Formatted at 2026-08-12 05:16:06 on dist-test-slave-srgv"
server_key: "23bc6b45fe101e5f57f3de07b0fe13bf"
server_key_iv: "6e5a0f69e21f443309a5d763a5b44195"
server_key_version: "encryptionkey@0"
I20260812 05:16:06.026950 27368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:06.042254 27368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:06.044348 27368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:06.044526 27368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:06.045054 27368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:06.045104 27368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.045130 27368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:06.045181 27368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.052284 27368 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43059
I20260812 05:16:06.052366 27505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43059 every 8 connection(s)
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511766, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:06.052778 27368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:06.053229 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 27368
I20260812 05:16:06.053351 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:16:06.053606 19099 external_mini_cluster.cc:1428] Setting key 0996416fd43a34757dd9f42d9ad43995
I20260812 05:16:06.055979 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.045625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:34493 (local address 127.18.166.254:37715)
0812 05:16:06.045701 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.045704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.045911 (+ 207us) server_negotiation.cc:407] Connection header received
0812 05:16:06.046399 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.046400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.046416 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.046498 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.047360 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.048200 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.049820 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.050165 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.050257 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.053154 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:06.053156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:06.053156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:06.053166 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:06.054861 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.055224 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.055224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.055225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.055275 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.055451 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.055452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.055452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.055558 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:06.055609 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.055850 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.055914 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:16:06.056125 27392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.045593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.18.166.195:34493)
0812 05:16:06.045766 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.045809 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.046298 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.046566 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.046569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.046945 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:06.047238 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.047246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.048302 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.048303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.049721 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.049725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.050270 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.050271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.050412 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.051013 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:06.051025 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:06.052998 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:06.054931 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:06.054937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:06.054939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:06.055158 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:06.055318 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:06.055318 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:06.055318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:06.055401 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:06.055617 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:06.055621 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:06.055737 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.055821 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.055940 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260812 05:16:06.056643 27506 heartbeater.cc:347] Connected to a master server at 127.18.166.254:37715
I20260812 05:16:06.056726 27506 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:06.056907 27506 heartbeater.cc:511] Master 127.18.166.254:37715 requested a full tablet report, sending...
I20260812 05:16:06.057283 26997 ts_manager.cc:194] Registered new tserver with Master: a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059)
I20260812 05:16:06.057811 26997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:34493
I20260812 05:16:06.062896 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.057729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:55067 (local address 127.18.166.254:37715)
0812 05:16:06.057951 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.057953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.058096 (+ 143us) server_negotiation.cc:407] Connection header received
0812 05:16:06.058181 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.058182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.058193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.058247 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.058545 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.059045 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.059682 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.059859 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.059906 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.060670 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:06.060671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:06.060672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:06.060678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:06.062037 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.062332 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.062333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.062333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.062382 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.062544 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.062544 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.062545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.062634 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:06.062683 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.062784 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.062837 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260812 05:16:06.062922 27513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.057650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.18.166.195:55067)
0812 05:16:06.058037 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.058049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.058111 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.058251 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.058251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.058415 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:06.058470 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.058471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.059132 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.059132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.059614 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.059616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.059921 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.059922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.059968 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.060327 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:06.060331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:06.060596 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:06.062106 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:06.062107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:06.062107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:06.062266 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:06.062439 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:06.062439 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:06.062440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:06.062491 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:06.062685 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:06.062686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:06.062720 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.062770 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.062853 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":69,"threads_started":1}
I20260812 05:16:06.065265 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511766, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 05:16:06.086510 27047 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:06.081164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.18.166.254:37715)
0812 05:16:06.081235 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.081237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.081246 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:06.081310 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.081310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.081320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.081372 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.081976 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.082461 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.083155 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.083331 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.083379 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.086389 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50114: BlockingRecv error: recv got EOF from 127.0.0.1:50114 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511766, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:06.094112 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.087295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50120 (local address 127.18.166.254:37715)
0812 05:16:06.087362 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.087364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.087453 (+ 89us) server_negotiation.cc:407] Connection header received
0812 05:16:06.087542 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.087543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.087553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.087602 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.088090 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.088617 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.089371 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.089617 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.089690 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.091935 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:06.091936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:06.091937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:06.091943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:06.093118 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.093463 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.093465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.093465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.093523 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.093697 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.093698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.093698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.093806 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:06.093883 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.093965 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.094021 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:16:06.095702 26997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:16:06.097289 26997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:06.103716 27529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.099559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:46739 (local address 127.0.0.1:59518)
0812 05:16:06.099829 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.099838 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.099881 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.100219 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.100222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.100224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.100302 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.100304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.101797 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.101798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.102656 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.102658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.103491 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.103492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.103552 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.103563 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.103621 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.103658 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":61,"threads_started":1}
I20260812 05:16:06.103811 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.099664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59518 (local address 127.18.166.194:46739)
0812 05:16:06.100055 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.100061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.100071 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:06.100112 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.100115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.100132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.100204 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.100592 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.101737 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.102753 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.103409 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.103504 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.103558 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.103655 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.103692 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":89,"threads_started":1}
I20260812 05:16:06.104442 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.099784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:33963 (local address 127.0.0.1:44102)
0812 05:16:06.099887 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.099899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.099951 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.100406 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.100407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.100409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.100453 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.100455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.101705 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.101706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.102957 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.102961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.104206 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.104208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.104269 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.104273 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.104330 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.104375 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":37,"thread_start_us":60,"threads_started":1}
I20260812 05:16:06.104508 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.100016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.18.166.193:33963)
0812 05:16:06.100228 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.100233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.100245 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:16:06.100284 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.100286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.100303 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.100408 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.100511 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.101640 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.103048 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.103772 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.103866 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.103922 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.104367 (+ 445us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.104404 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260812 05:16:06.104792 27290 tablet_service.cc:1467] Processing CreateTablet for tablet d2bffa6ecafc41b0bdb311f07878c789 (DEFAULT_TABLE table=test-table [id=a0a56020ed3042c5ad8353a0aba43dcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.104972 27530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.099554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43059 (local address 127.0.0.1:55130)
0812 05:16:06.099739 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.099793 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.099867 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.100506 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.100507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.100509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.100564 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.100567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.101617 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.101618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.102959 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.102961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.104726 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.104728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.104792 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.104796 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.104857 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.104898 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":37,"thread_start_us":87,"threads_started":1}
I20260812 05:16:06.105218 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2bffa6ecafc41b0bdb311f07878c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.105369 27143 tablet_service.cc:1467] Processing CreateTablet for tablet d2bffa6ecafc41b0bdb311f07878c789 (DEFAULT_TABLE table=test-table [id=a0a56020ed3042c5ad8353a0aba43dcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.105803 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2bffa6ecafc41b0bdb311f07878c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.108568 27535 tablet_bootstrap.cc:492] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Bootstrap starting.
I20260812 05:16:06.108896 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.099663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.18.166.195:43059)
0812 05:16:06.100336 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.100340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.100349 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:06.100387 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.100388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.100404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.100646 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.100690 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.101538 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.103969 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.104587 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.104733 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.104790 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.104870 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.104906 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":76,"threads_started":1}
I20260812 05:16:06.109093 27536 tablet_bootstrap.cc:492] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Bootstrap starting.
I20260812 05:16:06.109267 27535 tablet_bootstrap.cc:654] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.109645 27535 log.cc:824] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:06.109592 27434 tablet_service.cc:1467] Processing CreateTablet for tablet d2bffa6ecafc41b0bdb311f07878c789 (DEFAULT_TABLE table=test-table [id=a0a56020ed3042c5ad8353a0aba43dcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.109772 27536 tablet_bootstrap.cc:654] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.109941 27434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2bffa6ecafc41b0bdb311f07878c789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.110095 27536 log.cc:824] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:06.110495 27535 tablet_bootstrap.cc:492] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: No bootstrap required, opened a new log
I20260812 05:16:06.110571 27535 ts_tablet_manager.cc:1397] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:16:06.110853 27536 tablet_bootstrap.cc:492] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: No bootstrap required, opened a new log
I20260812 05:16:06.110941 27536 ts_tablet_manager.cc:1397] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:16:06.112391 27536 raft_consensus.cc:348] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.112542 27536 raft_consensus.cc:374] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.112586 27536 raft_consensus.cc:729] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8cd5633bc64091b0ff7b294d79bcfb, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.112727 27536 consensus_queue.cc:260] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.112661 27535 raft_consensus.cc:348] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.112829 27535 raft_consensus.cc:374] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.112884 27535 raft_consensus.cc:729] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.112993 27536 ts_tablet_manager.cc:1428] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:16:06.113031 27535 consensus_queue.cc:260] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.113114 27214 heartbeater.cc:503] Master 127.18.166.254:37715 was elected leader, sending a full tablet report...
I20260812 05:16:06.113307 27535 ts_tablet_manager.cc:1428] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:16:06.113356 27359 heartbeater.cc:503] Master 127.18.166.254:37715 was elected leader, sending a full tablet report...
I20260812 05:16:06.113688 27541 tablet_bootstrap.cc:492] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Bootstrap starting.
I20260812 05:16:06.114413 27541 tablet_bootstrap.cc:654] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.114781 27541 log.cc:824] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:06.115401 27541 tablet_bootstrap.cc:492] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: No bootstrap required, opened a new log
I20260812 05:16:06.115454 27541 ts_tablet_manager.cc:1397] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:16:06.116750 27541 raft_consensus.cc:348] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.116870 27541 raft_consensus.cc:374] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.116891 27541 raft_consensus.cc:729] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.117005 27541 consensus_queue.cc:260] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.117221 27541 ts_tablet_manager.cc:1428] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:16:06.117307 27506 heartbeater.cc:503] Master 127.18.166.254:37715 was elected leader, sending a full tablet report...
W20260812 05:16:06.136353 27360 tablet.cc:2369] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:16:06.140332 27540 raft_consensus.cc:482] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:06.140445 27540 raft_consensus.cc:504] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.140729 27540 leader_election.cc:288] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059), fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963)
I20260812 05:16:06.143738 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.141068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:35429 (local address 127.18.166.195:43059)
0812 05:16:06.141171 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.141189 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.141198 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:06.141240 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.141241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.141251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.141327 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.141537 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.142019 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.142968 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.143347 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.143436 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.143444 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.143622 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.143658 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 05:16:06.143764 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.140943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43059 (local address 127.18.166.194:35429)
0812 05:16:06.141006 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.141020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.141077 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.141333 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.141334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.141336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.141456 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.141458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.142108 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.142110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.142871 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.142873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.143436 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.143438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.143534 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.143537 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.143613 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.143655 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:16:06.143767 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.141161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:59519 (local address 127.18.166.193:33963)
0812 05:16:06.141226 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.141229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.141333 (+ 104us) server_negotiation.cc:407] Connection header received
0812 05:16:06.141439 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.141441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.141449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.141500 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.141656 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.142120 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.143033 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.143406 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.143494 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.143502 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.143657 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.143696 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:16:06.143791 27544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.141053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:33963 (local address 127.18.166.194:59519)
0812 05:16:06.141255 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.141264 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.141348 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.141503 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.141503 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.141505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.141586 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.141587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.142209 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.142210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.142939 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.142940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.143494 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.143496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.143580 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.143583 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.143651 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.143692 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":125,"threads_started":1}
I20260812 05:16:06.144222 27455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2bffa6ecafc41b0bdb311f07878c789" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06500afa15f4806b343abb20d1021fe" is_pre_election: true
I20260812 05:16:06.144384 27455 raft_consensus.cc:2454] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 0.
I20260812 05:16:06.144349 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2bffa6ecafc41b0bdb311f07878c789" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" is_pre_election: true
I20260812 05:16:06.144526 27164 raft_consensus.cc:2454] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 0.
I20260812 05:16:06.144670 27241 leader_election.cc:302] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, a06500afa15f4806b343abb20d1021fe; no voters:
I20260812 05:16:06.144814 27540 raft_consensus.cc:2781] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:06.144874 27540 raft_consensus.cc:482] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:06.144910 27540 raft_consensus.cc:3037] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.145977 27540 raft_consensus.cc:504] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.146117 27540 leader_election.cc:288] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 election: Requested vote from peers a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059), fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963)
I20260812 05:16:06.146330 27455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2bffa6ecafc41b0bdb311f07878c789" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06500afa15f4806b343abb20d1021fe"
I20260812 05:16:06.146354 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2bffa6ecafc41b0bdb311f07878c789" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
I20260812 05:16:06.146418 27164 raft_consensus.cc:3037] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.146430 27455 raft_consensus.cc:3037] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.147149 27164 raft_consensus.cc:2454] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 1.
I20260812 05:16:06.147161 27455 raft_consensus.cc:2454] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 1.
I20260812 05:16:06.147338 27241 leader_election.cc:302] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, fc8cd5633bc64091b0ff7b294d79bcfb; no voters:
I20260812 05:16:06.147475 27540 raft_consensus.cc:2781] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:06.147684 27540 raft_consensus.cc:686] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 1 LEADER]: Becoming Leader. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Running, Role: LEADER
I20260812 05:16:06.147799 27540 consensus_queue.cc:237] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } }
I20260812 05:16:06.148650 26996 catalog_manager.cc:5495] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6546b343d096402abf1916ec60b7aa6b (127.18.166.194). New cstate: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } health_report { overall_health: UNKNOWN } } }
I20260812 05:16:06.164301 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.162080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.18.166.194:46739)
0812 05:16:06.162141 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.162145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.162158 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:16:06.162192 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.162193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.162210 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.162274 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.162399 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.162863 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.163568 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.163751 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.163801 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.163961 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.164064 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.164123 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.164175 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.164192 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:16:06.166138 27164 raft_consensus.cc:1261] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Refusing update from remote peer 6546b343d096402abf1916ec60b7aa6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.166167 27455 raft_consensus.cc:1261] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Refusing update from remote peer 6546b343d096402abf1916ec60b7aa6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.166507 27540 consensus_queue.cc:1035] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.166656 27545 consensus_queue.cc:1035] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.169226 27550 mvcc.cc:205] Tried to move back new op lower bound from 7317552194213834752 to 7317552194142052352. Current Snapshot: MvccSnapshot[applied={T|T < 7317552194213834752}]
I20260812 05:16:06.169440 27548 mvcc.cc:205] Tried to move back new op lower bound from 7317552194213834752 to 7317552194142052352. Current Snapshot: MvccSnapshot[applied={T|T < 7317552194213834752}]
I20260812 05:16:06.171772 26996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:06.171867 26996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:06.172672 26996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Sending DeleteTablet for 3 replicas of tablet d2bffa6ecafc41b0bdb311f07878c789
I20260812 05:16:06.173038 27290 tablet_service.cc:1514] Processing DeleteTablet for tablet d2bffa6ecafc41b0bdb311f07878c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59518
I20260812 05:16:06.173059 27434 tablet_service.cc:1514] Processing DeleteTablet for tablet d2bffa6ecafc41b0bdb311f07878c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:55130
I20260812 05:16:06.173369 27143 tablet_service.cc:1514] Processing DeleteTablet for tablet d2bffa6ecafc41b0bdb311f07878c789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44102
I20260812 05:16:06.173508 27558 tablet_replica.cc:330] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: stopping tablet replica
I20260812 05:16:06.173504 26996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 05:16:06.173614 27558 raft_consensus.cc:2229] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 05:16:06.173648 26996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:06.174000 27558 raft_consensus.cc:2258] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.174125 27557 tablet_replica.cc:330] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: stopping tablet replica
I20260812 05:16:06.174243 27557 raft_consensus.cc:2229] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:06.174340 27557 raft_consensus.cc:2258] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.174815 27557 ts_tablet_manager.cc:1905] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.175124 27434 tablet_service.cc:1467] Processing CreateTablet for tablet 6e95f8feb98340d3abdf43be93af7a60 (DEFAULT_TABLE table=test-table [id=70f5d532fb5f40ee9c2d34450c3457bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.175266 27434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e95f8feb98340d3abdf43be93af7a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.176403 27559 tablet_replica.cc:330] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: stopping tablet replica
I20260812 05:16:06.176523 27559 raft_consensus.cc:2229] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:06.176534 27558 ts_tablet_manager.cc:1905] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.176669 27541 tablet_bootstrap.cc:492] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Bootstrap starting.
I20260812 05:16:06.176730 27557 ts_tablet_manager.cc:1918] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:06.176812 27557 log.cc:1196] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/d2bffa6ecafc41b0bdb311f07878c789
I20260812 05:16:06.177088 27557 ts_tablet_manager.cc:1939] T d2bffa6ecafc41b0bdb311f07878c789 P 6546b343d096402abf1916ec60b7aa6b: Deleting consensus metadata
I20260812 05:16:06.177096 27541 tablet_bootstrap.cc:654] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.177592 27143 tablet_service.cc:1467] Processing CreateTablet for tablet 6e95f8feb98340d3abdf43be93af7a60 (DEFAULT_TABLE table=test-table [id=70f5d532fb5f40ee9c2d34450c3457bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.177697 26983 catalog_manager.cc:4841] TS 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739): tablet d2bffa6ecafc41b0bdb311f07878c789 (table test-table [id=a0a56020ed3042c5ad8353a0aba43dcf]) successfully deleted
I20260812 05:16:06.177748 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e95f8feb98340d3abdf43be93af7a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.177983 27290 tablet_service.cc:1467] Processing CreateTablet for tablet 6e95f8feb98340d3abdf43be93af7a60 (DEFAULT_TABLE table=test-table [id=70f5d532fb5f40ee9c2d34450c3457bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.178097 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e95f8feb98340d3abdf43be93af7a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.178637 27558 ts_tablet_manager.cc:1918] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:06.178772 27558 log.cc:1196] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/d2bffa6ecafc41b0bdb311f07878c789
I20260812 05:16:06.179091 27558 ts_tablet_manager.cc:1939] T d2bffa6ecafc41b0bdb311f07878c789 P a06500afa15f4806b343abb20d1021fe: Deleting consensus metadata
I20260812 05:16:06.179307 27535 tablet_bootstrap.cc:492] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Bootstrap starting.
I20260812 05:16:06.179586 27536 tablet_bootstrap.cc:492] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Bootstrap starting.
I20260812 05:16:06.179576 26982 catalog_manager.cc:4841] TS a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059): tablet d2bffa6ecafc41b0bdb311f07878c789 (table test-table [id=a0a56020ed3042c5ad8353a0aba43dcf]) successfully deleted
I20260812 05:16:06.179733 27535 tablet_bootstrap.cc:654] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.179798 27541 tablet_bootstrap.cc:492] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: No bootstrap required, opened a new log
I20260812 05:16:06.179857 27541 ts_tablet_manager.cc:1397] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:16:06.179998 27536 tablet_bootstrap.cc:654] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.180016 27541 raft_consensus.cc:348] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.180090 27541 raft_consensus.cc:374] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.180115 27541 raft_consensus.cc:729] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.180160 27541 consensus_queue.cc:260] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.180253 27541 ts_tablet_manager.cc:1428] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:06.177583 27559 raft_consensus.cc:2258] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.180771 27535 tablet_bootstrap.cc:492] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: No bootstrap required, opened a new log
I20260812 05:16:06.180831 27535 ts_tablet_manager.cc:1397] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:16:06.180928 27536 tablet_bootstrap.cc:492] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: No bootstrap required, opened a new log
I20260812 05:16:06.180971 27535 raft_consensus.cc:348] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.181001 27536 ts_tablet_manager.cc:1397] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:06.181056 27535 raft_consensus.cc:374] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.181082 27535 raft_consensus.cc:729] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.181136 27535 consensus_queue.cc:260] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.181169 27536 raft_consensus.cc:348] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.181262 27536 raft_consensus.cc:374] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.181247 27535 ts_tablet_manager.cc:1428] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.181288 27536 raft_consensus.cc:729] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8cd5633bc64091b0ff7b294d79bcfb, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.181334 27536 consensus_queue.cc:260] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.181576 27536 ts_tablet_manager.cc:1428] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:06.181766 27559 ts_tablet_manager.cc:1905] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.183156 27559 ts_tablet_manager.cc:1918] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:06.183256 27559 log.cc:1196] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/d2bffa6ecafc41b0bdb311f07878c789
I20260812 05:16:06.183537 27559 ts_tablet_manager.cc:1939] T d2bffa6ecafc41b0bdb311f07878c789 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting consensus metadata
I20260812 05:16:06.183969 26982 catalog_manager.cc:4841] TS fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963): tablet d2bffa6ecafc41b0bdb311f07878c789 (table test-table [id=a0a56020ed3042c5ad8353a0aba43dcf]) successfully deleted
W20260812 05:16:06.218432 27216 tablet.cc:2369] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:16:06.303377 27507 tablet.cc:2369] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:16:06.379263 27539 raft_consensus.cc:482] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:06.379381 27539 raft_consensus.cc:504] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.379693 27539 leader_election.cc:288] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739), a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059)
I20260812 05:16:06.382766 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.379994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:51335 (local address 127.18.166.194:46739)
0812 05:16:06.380064 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.380067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.380115 (+ 48us) server_negotiation.cc:407] Connection header received
0812 05:16:06.380254 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.380255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.380268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.380326 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.380535 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.381001 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.382018 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.382384 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.382480 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.382490 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.382642 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.382680 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 05:16:06.382782 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.380074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:39059 (local address 127.18.166.195:43059)
0812 05:16:06.380133 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.380135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.380176 (+ 41us) server_negotiation.cc:407] Connection header received
0812 05:16:06.380252 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.380253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.380260 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.380322 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.380504 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.380981 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.382053 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.382409 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.382496 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.382504 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.382668 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.382704 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:16:06.382818 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.379900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43059 (local address 127.18.166.193:39059)
0812 05:16:06.380113 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.380123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.380187 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.380351 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.380352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.380355 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.380434 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.380437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.381079 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.381081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.381958 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.381961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.382497 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.382498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.382580 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.382583 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.382660 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.382704 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":73,"thread_start_us":61,"threads_started":1}
I20260812 05:16:06.382822 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.379900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:46739 (local address 127.18.166.193:51335)
0812 05:16:06.380035 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.380048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.380150 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.380395 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.380396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.380398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.380449 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.380451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.381099 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.381101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.381920 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.381922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.382482 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.382483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.382565 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.382568 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.382636 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.382680 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260812 05:16:06.383155 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95f8feb98340d3abdf43be93af7a60" candidate_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6546b343d096402abf1916ec60b7aa6b" is_pre_election: true
I20260812 05:16:06.383280 27310 raft_consensus.cc:2454] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8cd5633bc64091b0ff7b294d79bcfb in term 0.
I20260812 05:16:06.383368 27455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95f8feb98340d3abdf43be93af7a60" candidate_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06500afa15f4806b343abb20d1021fe" is_pre_election: true
I20260812 05:16:06.383459 27455 raft_consensus.cc:2454] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8cd5633bc64091b0ff7b294d79bcfb in term 0.
I20260812 05:16:06.383467 27092 leader_election.cc:302] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, fc8cd5633bc64091b0ff7b294d79bcfb; no voters:
I20260812 05:16:06.383612 27539 raft_consensus.cc:2781] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:06.383651 27539 raft_consensus.cc:482] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:06.383675 27539 raft_consensus.cc:3037] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.384346 27539 raft_consensus.cc:504] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.384492 27539 leader_election.cc:288] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [CANDIDATE]: Term 1 election: Requested vote from peers 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739), a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059)
I20260812 05:16:06.384642 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95f8feb98340d3abdf43be93af7a60" candidate_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6546b343d096402abf1916ec60b7aa6b"
I20260812 05:16:06.384644 27455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e95f8feb98340d3abdf43be93af7a60" candidate_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06500afa15f4806b343abb20d1021fe"
I20260812 05:16:06.384727 27310 raft_consensus.cc:3037] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.384727 27455 raft_consensus.cc:3037] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.385521 27455 raft_consensus.cc:2454] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8cd5633bc64091b0ff7b294d79bcfb in term 1.
I20260812 05:16:06.385586 27310 raft_consensus.cc:2454] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8cd5633bc64091b0ff7b294d79bcfb in term 1.
I20260812 05:16:06.385725 27092 leader_election.cc:302] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, fc8cd5633bc64091b0ff7b294d79bcfb; no voters:
I20260812 05:16:06.385867 27539 raft_consensus.cc:2781] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:06.386041 27539 raft_consensus.cc:686] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 LEADER]: Becoming Leader. State: Replica: fc8cd5633bc64091b0ff7b294d79bcfb, State: Running, Role: LEADER
I20260812 05:16:06.386116 27539 consensus_queue.cc:237] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.386869 26996 catalog_manager.cc:5495] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193). New cstate: current_term: 1 leader_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } health_report { overall_health: UNKNOWN } } }
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511765, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.254@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:06.440626 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.434164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:37715 (local address 127.0.0.1:35940)
0812 05:16:06.434322 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.434333 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.434401 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.434522 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.434522 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.434828 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:06.434878 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.434880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.435502 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.435503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.436014 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.436015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.436327 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.436328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.436368 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.436846 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:06.436850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:06.438349 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:06.439718 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:06.439726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:06.439727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:06.439917 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:06.440095 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:06.440095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:06.440096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:06.440149 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:06.440364 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:06.440370 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:06.440411 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.440458 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.440548 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260812 05:16:06.440680 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.434245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.18.166.254:37715)
0812 05:16:06.434333 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.434337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.434396 (+ 59us) server_negotiation.cc:407] Connection header received
0812 05:16:06.434453 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.434454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.434464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.434515 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.434940 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.435428 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.436081 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.436255 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.436311 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.438432 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:06.438433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:06.438434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:06.438440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:06.439644 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.439984 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.439985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.439985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.440035 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.440202 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.440202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.440203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.440309 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:06.440360 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.440483 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.440594 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":59}
I20260812 05:16:06.442830 26996 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:35940:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 05:16:06.443041 26996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:06.444469 27290 tablet_service.cc:1467] Processing CreateTablet for tablet 85b8243d2cd3412588f46060473239bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=114f9bd498224cb59dafad4f68b8e5c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:16:06.444527 27434 tablet_service.cc:1467] Processing CreateTablet for tablet 85b8243d2cd3412588f46060473239bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=114f9bd498224cb59dafad4f68b8e5c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:16:06.444506 27143 tablet_service.cc:1467] Processing CreateTablet for tablet 85b8243d2cd3412588f46060473239bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=114f9bd498224cb59dafad4f68b8e5c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:16:06.444664 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b8243d2cd3412588f46060473239bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.444665 27434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b8243d2cd3412588f46060473239bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.444666 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b8243d2cd3412588f46060473239bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.446081 27535 tablet_bootstrap.cc:492] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: Bootstrap starting.
I20260812 05:16:06.446245 27536 tablet_bootstrap.cc:492] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: Bootstrap starting.
I20260812 05:16:06.446261 27541 tablet_bootstrap.cc:492] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: Bootstrap starting.
I20260812 05:16:06.446496 27535 tablet_bootstrap.cc:654] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.446691 27536 tablet_bootstrap.cc:654] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.446691 27541 tablet_bootstrap.cc:654] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.447431 27536 tablet_bootstrap.cc:492] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: No bootstrap required, opened a new log
I20260812 05:16:06.447511 27536 ts_tablet_manager.cc:1397] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:06.447557 27535 tablet_bootstrap.cc:492] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: No bootstrap required, opened a new log
I20260812 05:16:06.447615 27535 ts_tablet_manager.cc:1397] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:16:06.447677 27536 raft_consensus.cc:348] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.447736 27535 raft_consensus.cc:348] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.447769 27536 raft_consensus.cc:374] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.447798 27536 raft_consensus.cc:729] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8cd5633bc64091b0ff7b294d79bcfb, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.447785 27535 raft_consensus.cc:374] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.447825 27535 raft_consensus.cc:729] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.447851 27536 consensus_queue.cc:260] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.447876 27535 consensus_queue.cc:260] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.447950 27536 ts_tablet_manager.cc:1428] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:06.447960 27535 ts_tablet_manager.cc:1428] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 05:16:06.448001 27539 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.448056 27545 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.448118 27545 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.448113 27539 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.448194 27541 tablet_bootstrap.cc:492] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: No bootstrap required, opened a new log
I20260812 05:16:06.448243 27541 ts_tablet_manager.cc:1397] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:16:06.448412 27541 raft_consensus.cc:348] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.448486 27541 raft_consensus.cc:374] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.448511 27541 raft_consensus.cc:729] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.448558 27541 consensus_queue.cc:260] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.448664 27541 ts_tablet_manager.cc:1428] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:06.448809 27543 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.448892 27543 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.469242 27545 raft_consensus.cc:482] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:06.469368 27545 raft_consensus.cc:504] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.469594 27545 leader_election.cc:288] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963), a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059)
I20260812 05:16:06.469903 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b8243d2cd3412588f46060473239bd" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" is_pre_election: true
I20260812 05:16:06.469903 27455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b8243d2cd3412588f46060473239bd" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06500afa15f4806b343abb20d1021fe" is_pre_election: true
I20260812 05:16:06.470018 27164 raft_consensus.cc:2454] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 0.
I20260812 05:16:06.470018 27455 raft_consensus.cc:2454] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 0.
I20260812 05:16:06.470187 27241 leader_election.cc:302] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, fc8cd5633bc64091b0ff7b294d79bcfb; no voters:
I20260812 05:16:06.470297 27545 raft_consensus.cc:2781] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:06.470346 27545 raft_consensus.cc:482] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:06.470367 27545 raft_consensus.cc:3037] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.471095 27545 raft_consensus.cc:504] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.471233 27545 leader_election.cc:288] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 election: Requested vote from peers fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963), a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059)
I20260812 05:16:06.471467 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b8243d2cd3412588f46060473239bd" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
I20260812 05:16:06.471463 27455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b8243d2cd3412588f46060473239bd" candidate_uuid: "6546b343d096402abf1916ec60b7aa6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06500afa15f4806b343abb20d1021fe"
I20260812 05:16:06.471547 27164 raft_consensus.cc:3037] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.471547 27455 raft_consensus.cc:3037] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.472306 27455 raft_consensus.cc:2454] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 1.
I20260812 05:16:06.472314 27164 raft_consensus.cc:2454] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6546b343d096402abf1916ec60b7aa6b in term 1.
I20260812 05:16:06.472503 27241 leader_election.cc:302] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, a06500afa15f4806b343abb20d1021fe; no voters:
I20260812 05:16:06.472645 27545 raft_consensus.cc:2781] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:06.472713 27545 raft_consensus.cc:686] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [term 1 LEADER]: Becoming Leader. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Running, Role: LEADER
I20260812 05:16:06.472770 27545 consensus_queue.cc:237] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.473085 27540 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: TxnStatusTablet state changed. Reason: New leader 6546b343d096402abf1916ec60b7aa6b. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.473166 27540 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:16:06.473372 26996 catalog_manager.cc:5495] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6546b343d096402abf1916ec60b7aa6b (127.18.166.194). New cstate: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } health_report { overall_health: UNKNOWN } } }
I20260812 05:16:06.473465 27577 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 05:16:06.473574 27577 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 05:16:06.486572 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.483870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.18.166.194:46739)
0812 05:16:06.483954 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.483957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.483979 (+ 22us) server_negotiation.cc:407] Connection header received
0812 05:16:06.484007 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.484008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.484020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.484086 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.484219 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.484963 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.485775 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.486079 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.486152 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.486330 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.486367 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.486416 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.486470 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.486493 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:16:06.486585 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.483784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:46739 (local address 127.0.0.1:58548)
0812 05:16:06.483867 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.483882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.483939 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.484094 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.484094 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.484095 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:16:06.484151 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.484153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.485060 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.485061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.485711 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.485713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.486152 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.486153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.486202 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.486218 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.486423 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.486430 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.486467 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.486506 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:16:06.487669 27455 raft_consensus.cc:1261] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Refusing update from remote peer 6546b343d096402abf1916ec60b7aa6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.487715 27164 raft_consensus.cc:1261] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Refusing update from remote peer 6546b343d096402abf1916ec60b7aa6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.487912 27540 consensus_queue.cc:1035] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.488036 27545 consensus_queue.cc:1035] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.488592 27539 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: TxnStatusTablet state changed. Reason: New leader 6546b343d096402abf1916ec60b7aa6b. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.488601 27543 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: TxnStatusTablet state changed. Reason: New leader 6546b343d096402abf1916ec60b7aa6b. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.488683 27539 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.488682 27543 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.489755 27540 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.489846 27540 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:16:06.490046 27549 mvcc.cc:205] Tried to move back new op lower bound from 7317552195531800576 to 7317552195472924672. Current Snapshot: MvccSnapshot[applied={T|T < 7317552195531800576}]
I20260812 05:16:06.489761 27545 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.490370 27545 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P 6546b343d096402abf1916ec60b7aa6b: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:16:06.490878 27539 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.490971 27539 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P fc8cd5633bc64091b0ff7b294d79bcfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.491359 27543 tablet_replica.cc:439] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6546b343d096402abf1916ec60b7aa6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } } }
I20260812 05:16:06.491456 27543 tablet_replica.cc:442] T 85b8243d2cd3412588f46060473239bd P a06500afa15f4806b343abb20d1021fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:16:06.494176 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.491796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38496 (local address 127.18.166.193:33963)
0812 05:16:06.491833 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.491836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.491867 (+ 31us) server_negotiation.cc:407] Connection header received
0812 05:16:06.491949 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.491950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.491970 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.492033 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.492150 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.492622 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.493369 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.493613 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.493693 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.493841 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.493950 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.494029 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.494077 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.494098 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:16:06.499272 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.496622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:46739 (local address 127.18.166.193:57053)
0812 05:16:06.496790 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.496801 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.496852 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.496969 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.496969 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.496970 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:16:06.497022 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.497023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.497899 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.497899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.498461 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.498463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.498913 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.498914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.498958 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.498972 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.499138 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.499140 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.499174 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.499204 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260812 05:16:06.499274 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.496698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:57053 (local address 127.18.166.194:46739)
0812 05:16:06.496784 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.496786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.496862 (+ 76us) server_negotiation.cc:407] Connection header received
0812 05:16:06.496888 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.496889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.496902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.496965 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.497090 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.497827 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.498541 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.498837 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.498907 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.499047 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.499078 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.499131 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.499177 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.499196 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 05:16:06.501526 27455 raft_consensus.cc:1261] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Refusing update from remote peer fc8cd5633bc64091b0ff7b294d79bcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.501616 27310 raft_consensus.cc:1261] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Refusing update from remote peer fc8cd5633bc64091b0ff7b294d79bcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.501868 27539 consensus_queue.cc:1035] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.502362 27565 consensus_queue.cc:1035] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.511178 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.508309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:57591 (local address 127.18.166.193:33963)
0812 05:16:06.508353 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.508355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.508474 (+ 119us) server_negotiation.cc:407] Connection header received
0812 05:16:06.508548 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.508549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.508560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.508616 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.508846 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.509461 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.510482 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.510688 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.510771 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.510939 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.510972 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.511026 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.511084 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.511105 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:16:06.511200 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.508250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:33963 (local address 127.18.166.194:57591)
0812 05:16:06.508415 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.508427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.508488 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.508736 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.508737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.508738 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:16:06.508784 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.508786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.509569 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.509570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.510394 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.510397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.510773 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.510774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.510831 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.510847 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.511029 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.511032 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.511072 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.511118 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260812 05:16:06.523940 26997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:06.524078 26997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:06.524731 26997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Sending DeleteTablet for 3 replicas of tablet 6e95f8feb98340d3abdf43be93af7a60
I20260812 05:16:06.525017 27143 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e95f8feb98340d3abdf43be93af7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44102
I20260812 05:16:06.525134 27559 tablet_replica.cc:330] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: stopping tablet replica
I20260812 05:16:06.525136 27434 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e95f8feb98340d3abdf43be93af7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:55130
I20260812 05:16:06.525210 27559 raft_consensus.cc:2229] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:06.525238 27558 tablet_replica.cc:330] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: stopping tablet replica
I20260812 05:16:06.525280 27558 raft_consensus.cc:2229] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:06.525306 27559 raft_consensus.cc:2258] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.525315 27558 raft_consensus.cc:2258] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.525398 27290 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e95f8feb98340d3abdf43be93af7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59518
I20260812 05:16:06.525643 27557 tablet_replica.cc:330] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: stopping tablet replica
I20260812 05:16:06.525698 27557 raft_consensus.cc:2229] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:06.525692 27559 ts_tablet_manager.cc:1905] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.525734 27557 raft_consensus.cc:2258] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.526167 27557 ts_tablet_manager.cc:1905] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.527452 27559 ts_tablet_manager.cc:1918] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:16:06.527506 27559 log.cc:1196] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/6e95f8feb98340d3abdf43be93af7a60
I20260812 05:16:06.527761 27559 ts_tablet_manager.cc:1939] T 6e95f8feb98340d3abdf43be93af7a60 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting consensus metadata
I20260812 05:16:06.527884 27557 ts_tablet_manager.cc:1918] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:16:06.527930 27557 log.cc:1196] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/6e95f8feb98340d3abdf43be93af7a60
I20260812 05:16:06.528129 27557 ts_tablet_manager.cc:1939] T 6e95f8feb98340d3abdf43be93af7a60 P 6546b343d096402abf1916ec60b7aa6b: Deleting consensus metadata
I20260812 05:16:06.528301 26982 catalog_manager.cc:4841] TS fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963): tablet 6e95f8feb98340d3abdf43be93af7a60 (table test-table [id=70f5d532fb5f40ee9c2d34450c3457bb]) successfully deleted
I20260812 05:16:06.528508 27558 ts_tablet_manager.cc:1905] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.528798 26983 catalog_manager.cc:4841] TS 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739): tablet 6e95f8feb98340d3abdf43be93af7a60 (table test-table [id=70f5d532fb5f40ee9c2d34450c3457bb]) successfully deleted
I20260812 05:16:06.530320 27558 ts_tablet_manager.cc:1918] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:16:06.530387 27558 log.cc:1196] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/6e95f8feb98340d3abdf43be93af7a60
I20260812 05:16:06.530618 27558 ts_tablet_manager.cc:1939] T 6e95f8feb98340d3abdf43be93af7a60 P a06500afa15f4806b343abb20d1021fe: Deleting consensus metadata
I20260812 05:16:06.530941 26982 catalog_manager.cc:4841] TS a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059): tablet 6e95f8feb98340d3abdf43be93af7a60 (table test-table [id=70f5d532fb5f40ee9c2d34450c3457bb]) successfully deleted
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 05:16:06.558593 27047 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:06.551116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35944 (local address 127.18.166.254:37715)
0812 05:16:06.551166 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.551169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.551351 (+ 182us) server_negotiation.cc:407] Connection header received
0812 05:16:06.551396 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.551397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.551410 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.551478 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.552254 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.553237 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.554295 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.554609 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.554687 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.558520 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35944: BlockingRecv error: recv got EOF from 127.0.0.1:35944 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 05:16:06 dist-test-slave-srgv krb5kdc[26951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM
I20260812 05:16:06.571010 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.561701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.18.166.254:37715)
0812 05:16:06.561745 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.561764 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.561908 (+ 144us) server_negotiation.cc:407] Connection header received
0812 05:16:06.561955 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.561956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.561968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.562047 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:06.562657 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.563415 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.564495 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.564783 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.564865 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.567749 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:06.567750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:06.567751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:06.567759 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:06.569579 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.569994 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.569995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.569996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.570062 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:06.570337 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:06.570337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:06.570338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:06.570479 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:06.570579 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.570810 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.570897 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:16:06.572801 26997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:16:06.572985 26997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:06.574536 27290 tablet_service.cc:1467] Processing CreateTablet for tablet 8003b10e67fc42928ae4768ae1064b99 (DEFAULT_TABLE table=test-table [id=795eb43dc7db4cd591cd0f5d3fa33d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.574581 27143 tablet_service.cc:1467] Processing CreateTablet for tablet 8003b10e67fc42928ae4768ae1064b99 (DEFAULT_TABLE table=test-table [id=795eb43dc7db4cd591cd0f5d3fa33d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.574692 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8003b10e67fc42928ae4768ae1064b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.574723 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8003b10e67fc42928ae4768ae1064b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.576082 27536 tablet_bootstrap.cc:492] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Bootstrap starting.
I20260812 05:16:06.576227 27434 tablet_service.cc:1467] Processing CreateTablet for tablet 8003b10e67fc42928ae4768ae1064b99 (DEFAULT_TABLE table=test-table [id=795eb43dc7db4cd591cd0f5d3fa33d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.576360 27434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8003b10e67fc42928ae4768ae1064b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.576538 27536 tablet_bootstrap.cc:654] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.577538 27535 tablet_bootstrap.cc:492] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Bootstrap starting.
I20260812 05:16:06.577764 27541 tablet_bootstrap.cc:492] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Bootstrap starting.
I20260812 05:16:06.577889 27535 tablet_bootstrap.cc:654] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.578179 27541 tablet_bootstrap.cc:654] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.579226 27536 tablet_bootstrap.cc:492] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: No bootstrap required, opened a new log
I20260812 05:16:06.579262 27535 tablet_bootstrap.cc:492] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: No bootstrap required, opened a new log
I20260812 05:16:06.579279 27536 ts_tablet_manager.cc:1397] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:16:06.579308 27535 ts_tablet_manager.cc:1397] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:16:06.579429 27536 raft_consensus.cc:348] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.579499 27536 raft_consensus.cc:374] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.579456 27535 raft_consensus.cc:348] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.579519 27536 raft_consensus.cc:729] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8cd5633bc64091b0ff7b294d79bcfb, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.579525 27535 raft_consensus.cc:374] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.579547 27535 raft_consensus.cc:729] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.579562 27536 consensus_queue.cc:260] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.579612 27535 consensus_queue.cc:260] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.579646 27536 ts_tablet_manager.cc:1428] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.579712 27535 ts_tablet_manager.cc:1428] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 05:16:06.580931 27541 tablet_bootstrap.cc:492] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: No bootstrap required, opened a new log
I20260812 05:16:06.580996 27541 ts_tablet_manager.cc:1397] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:16:06.581157 27541 raft_consensus.cc:348] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.581249 27541 raft_consensus.cc:374] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.581272 27541 raft_consensus.cc:729] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.581318 27541 consensus_queue.cc:260] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.581472 27541 ts_tablet_manager.cc:1428] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:16:06.592834 27543 raft_consensus.cc:482] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:06.592953 27543 raft_consensus.cc:504] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.593325 27543 leader_election.cc:288] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963), 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739)
I20260812 05:16:06.597689 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.593783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:45049 (local address 127.18.166.194:46739)
0812 05:16:06.593828 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.593830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.593839 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:06.593869 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.593870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.593879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.593947 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.594132 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.594839 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.596674 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.597216 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.597264 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.597273 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.597566 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.597618 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:16:06.598141 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.593591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:46739 (local address 127.18.166.195:45049)
0812 05:16:06.593742 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.593755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.593814 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.593954 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.593955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.593958 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.594037 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.594039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.594944 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.594945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.596568 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.596571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.597346 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.597347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.597458 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.597462 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.597999 (+ 537us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.598049 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260812 05:16:06.598284 27534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.594255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:48299 (local address 127.18.166.193:33963)
0812 05:16:06.594300 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.594302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.594411 (+ 109us) server_negotiation.cc:407] Connection header received
0812 05:16:06.594502 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.594503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.594512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.594539 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:06.595078 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.595768 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.597246 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.597826 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.597892 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.597900 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.598177 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.598215 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:16:06.598619 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8003b10e67fc42928ae4768ae1064b99" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6546b343d096402abf1916ec60b7aa6b" is_pre_election: true
I20260812 05:16:06.598690 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.594088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:33963 (local address 127.18.166.195:48299)
0812 05:16:06.594340 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.594351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.594433 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.594598 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.594599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.594601 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:06.594972 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.594975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.595940 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.595941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.597139 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.597143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.598013 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.598014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.598084 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.598087 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.598567 (+ 480us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.598611 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":71,"threads_started":1}
I20260812 05:16:06.598718 27310 raft_consensus.cc:2454] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 0.
I20260812 05:16:06.598975 27386 leader_election.cc:302] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, a06500afa15f4806b343abb20d1021fe; no voters:
I20260812 05:16:06.599129 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8003b10e67fc42928ae4768ae1064b99" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" is_pre_election: true
I20260812 05:16:06.599228 27164 raft_consensus.cc:2454] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 0.
I20260812 05:16:06.599306 27543 raft_consensus.cc:2781] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:06.599349 27543 raft_consensus.cc:482] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:06.599373 27543 raft_consensus.cc:3037] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.600312 27543 raft_consensus.cc:504] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.600450 27543 leader_election.cc:288] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 election: Requested vote from peers fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963), 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739)
I20260812 05:16:06.600816 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8003b10e67fc42928ae4768ae1064b99" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
I20260812 05:16:06.600889 27164 raft_consensus.cc:3037] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.601639 27164 raft_consensus.cc:2454] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 1.
I20260812 05:16:06.601822 27385 leader_election.cc:302] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06500afa15f4806b343abb20d1021fe, fc8cd5633bc64091b0ff7b294d79bcfb; no voters:
I20260812 05:16:06.601925 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8003b10e67fc42928ae4768ae1064b99" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6546b343d096402abf1916ec60b7aa6b"
I20260812 05:16:06.602002 27310 raft_consensus.cc:3037] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.602747 27310 raft_consensus.cc:2454] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 1.
I20260812 05:16:06.602952 27543 raft_consensus.cc:2781] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:06.603089 27543 raft_consensus.cc:686] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 1 LEADER]: Becoming Leader. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Running, Role: LEADER
I20260812 05:16:06.603168 27543 consensus_queue.cc:237] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.603840 26997 catalog_manager.cc:5495] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe reported cstate change: term changed from 0 to 1, leader changed from <none> to a06500afa15f4806b343abb20d1021fe (127.18.166.195). New cstate: current_term: 1 leader_uuid: "a06500afa15f4806b343abb20d1021fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } health_report { overall_health: HEALTHY } } }
I20260812 05:16:06.612756 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.609642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56206 (local address 127.18.166.195:43059)
0812 05:16:06.609694 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.609696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.609734 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:16:06.609775 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.609776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.609797 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.609869 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.610047 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.610776 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.611839 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.612118 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.612210 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.612390 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.612511 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.612600 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.612655 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.612676 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:16:06.614601 27164 raft_consensus.cc:1261] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Refusing update from remote peer a06500afa15f4806b343abb20d1021fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.614974 27613 consensus_queue.cc:1035] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.615458 27310 raft_consensus.cc:1261] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Refusing update from remote peer a06500afa15f4806b343abb20d1021fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.615705 27613 consensus_queue.cc:1035] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.619328 26997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:06.619431 26997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:06.620636 26997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Sending DeleteTablet for 3 replicas of tablet 8003b10e67fc42928ae4768ae1064b99
I20260812 05:16:06.620962 27143 tablet_service.cc:1514] Processing DeleteTablet for tablet 8003b10e67fc42928ae4768ae1064b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44102
I20260812 05:16:06.621068 27434 tablet_service.cc:1514] Processing DeleteTablet for tablet 8003b10e67fc42928ae4768ae1064b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:55130
I20260812 05:16:06.621162 27559 tablet_replica.cc:330] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: stopping tablet replica
I20260812 05:16:06.621223 27559 raft_consensus.cc:2229] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:06.621269 27559 raft_consensus.cc:2258] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.621342 27558 tablet_replica.cc:330] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: stopping tablet replica
I20260812 05:16:06.621431 27558 raft_consensus.cc:2229] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:06.621511 27558 raft_consensus.cc:2258] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.621902 27290 tablet_service.cc:1514] Processing DeleteTablet for tablet 8003b10e67fc42928ae4768ae1064b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:06 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59518
I20260812 05:16:06.622017 27557 tablet_replica.cc:330] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: stopping tablet replica
I20260812 05:16:06.622092 27557 raft_consensus.cc:2229] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:06.622140 27557 pending_rounds.cc:62] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Trying to abort 2 pending ops.
I20260812 05:16:06.622169 27557 pending_rounds.cc:68] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317552196050644992 op_type: WRITE_OP write_request { tablet_id: "8003b10e67fc42928ae4768ae1064b99" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\302\205\360\323\006\"8\n\ntest-admin\022*\n 795eb43dc7db4cd591cd0f5d3fa33d5e\020\001\030\001 \001(\001" signature: "<redacted>""l\270\226+\242\255d\223\t\274\031V\354R\344MV\234 \367*\207\277\352\273\214\321\206v)e\310\205\342}\322:>\034P\331\035\367\312\027\207\030B\301\203\3665hO\303!\257-\255,\235r\331]" signing_key_seq_num: 0 } } request_id { client_id: "3bc56e3b97fb4a7495a7da7e2f72bd86" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 05:16:06.622347 27557 pending_rounds.cc:68] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317552196007096320 op_type: NO_OP noop_request { }
I20260812 05:16:06.622391 27557 raft_consensus.cc:2866] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260812 05:16:06.622426 27557 raft_consensus.cc:2258] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:06.622462 27558 ts_tablet_manager.cc:1905] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.624190 27558 ts_tablet_manager.cc:1918] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:06.624260 27558 log.cc:1196] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/8003b10e67fc42928ae4768ae1064b99
I20260812 05:16:06.624390 26996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 05:16:06.624521 27558 ts_tablet_manager.cc:1939] T 8003b10e67fc42928ae4768ae1064b99 P a06500afa15f4806b343abb20d1021fe: Deleting consensus metadata
W20260812 05:16:06.624555 26996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:06.624861 26982 catalog_manager.cc:4841] TS a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059): tablet 8003b10e67fc42928ae4768ae1064b99 (table test-table [id=795eb43dc7db4cd591cd0f5d3fa33d5e]) successfully deleted
I20260812 05:16:06.625808 27434 tablet_service.cc:1467] Processing CreateTablet for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e (DEFAULT_TABLE table=test-table [id=e3086b4c54b2443c87ee8687cfcd97d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.625667 27559 ts_tablet_manager.cc:1905] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.625921 27434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.626961 27541 tablet_bootstrap.cc:492] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Bootstrap starting.
I20260812 05:16:06.627354 27541 tablet_bootstrap.cc:654] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.627355 27559 ts_tablet_manager.cc:1918] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:06.627413 27559 log.cc:1196] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/8003b10e67fc42928ae4768ae1064b99
I20260812 05:16:06.627647 27559 ts_tablet_manager.cc:1939] T 8003b10e67fc42928ae4768ae1064b99 P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting consensus metadata
I20260812 05:16:06.627866 27143 tablet_service.cc:1467] Processing CreateTablet for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e (DEFAULT_TABLE table=test-table [id=e3086b4c54b2443c87ee8687cfcd97d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.627970 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.628962 27536 tablet_bootstrap.cc:492] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Bootstrap starting.
I20260812 05:16:06.629367 27536 tablet_bootstrap.cc:654] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.629958 26982 catalog_manager.cc:4841] TS fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963): tablet 8003b10e67fc42928ae4768ae1064b99 (table test-table [id=795eb43dc7db4cd591cd0f5d3fa33d5e]) successfully deleted
I20260812 05:16:06.630336 27290 tablet_service.cc:1467] Processing CreateTablet for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e (DEFAULT_TABLE table=test-table [id=e3086b4c54b2443c87ee8687cfcd97d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:06.630452 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:06.630726 27541 tablet_bootstrap.cc:492] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: No bootstrap required, opened a new log
I20260812 05:16:06.630770 27541 ts_tablet_manager.cc:1397] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 05:16:06.630929 27541 raft_consensus.cc:348] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.630999 27541 raft_consensus.cc:374] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.631021 27541 raft_consensus.cc:729] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.631057 27541 consensus_queue.cc:260] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.631167 27541 ts_tablet_manager.cc:1428] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260812 05:16:06.631748 27618 consensus_queue.cc:1159] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6546b343d096402abf1916ec60b7aa6b" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260812 05:16:06.631757 27535 tablet_bootstrap.cc:492] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Bootstrap starting.
I20260812 05:16:06.632056 27557 ts_tablet_manager.cc:1905] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:06.632627 27535 tablet_bootstrap.cc:654] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:06.633513 27557 ts_tablet_manager.cc:1918] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:06.633574 27557 log.cc:1196] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/8003b10e67fc42928ae4768ae1064b99
I20260812 05:16:06.633805 27557 ts_tablet_manager.cc:1939] T 8003b10e67fc42928ae4768ae1064b99 P 6546b343d096402abf1916ec60b7aa6b: Deleting consensus metadata
I20260812 05:16:06.634150 26983 catalog_manager.cc:4841] TS 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739): tablet 8003b10e67fc42928ae4768ae1064b99 (table test-table [id=795eb43dc7db4cd591cd0f5d3fa33d5e]) successfully deleted
I20260812 05:16:06.634645 27536 tablet_bootstrap.cc:492] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: No bootstrap required, opened a new log
I20260812 05:16:06.634696 27536 ts_tablet_manager.cc:1397] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260812 05:16:06.634864 27535 tablet_bootstrap.cc:492] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: No bootstrap required, opened a new log
I20260812 05:16:06.634853 27536 raft_consensus.cc:348] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.634908 27535 ts_tablet_manager.cc:1397] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 05:16:06.634913 27536 raft_consensus.cc:374] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.634932 27536 raft_consensus.cc:729] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8cd5633bc64091b0ff7b294d79bcfb, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.634972 27536 consensus_queue.cc:260] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.635056 27536 ts_tablet_manager.cc:1428] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.635035 27535 raft_consensus.cc:348] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.635099 27535 raft_consensus.cc:374] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:06.635123 27535 raft_consensus.cc:729] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6546b343d096402abf1916ec60b7aa6b, State: Initialized, Role: FOLLOWER
I20260812 05:16:06.635165 27535 consensus_queue.cc:260] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.635257 27535 ts_tablet_manager.cc:1428] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:06.952389 27543 raft_consensus.cc:482] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:06.952497 27543 raft_consensus.cc:504] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.952701 27543 leader_election.cc:288] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963), 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739)
I20260812 05:16:06.952965 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad0356b8c5455e92bfcc1ae87b4f1e" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6546b343d096402abf1916ec60b7aa6b" is_pre_election: true
I20260812 05:16:06.952963 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad0356b8c5455e92bfcc1ae87b4f1e" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" is_pre_election: true
I20260812 05:16:06.953058 27164 raft_consensus.cc:2454] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 0.
I20260812 05:16:06.953075 27310 raft_consensus.cc:2454] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 0.
I20260812 05:16:06.953222 27386 leader_election.cc:302] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6546b343d096402abf1916ec60b7aa6b, a06500afa15f4806b343abb20d1021fe; no voters:
I20260812 05:16:06.953351 27543 raft_consensus.cc:2781] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:06.953390 27543 raft_consensus.cc:482] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:06.953457 27543 raft_consensus.cc:3037] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.954137 27543 raft_consensus.cc:504] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.954280 27543 leader_election.cc:288] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 election: Requested vote from peers fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963), 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739)
I20260812 05:16:06.954448 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad0356b8c5455e92bfcc1ae87b4f1e" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6546b343d096402abf1916ec60b7aa6b"
I20260812 05:16:06.954455 27164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bad0356b8c5455e92bfcc1ae87b4f1e" candidate_uuid: "a06500afa15f4806b343abb20d1021fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb"
I20260812 05:16:06.954524 27310 raft_consensus.cc:3037] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.954545 27164 raft_consensus.cc:3037] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:06.955133 27164 raft_consensus.cc:2454] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 1.
I20260812 05:16:06.955175 27310 raft_consensus.cc:2454] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06500afa15f4806b343abb20d1021fe in term 1.
I20260812 05:16:06.955340 27385 leader_election.cc:302] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06500afa15f4806b343abb20d1021fe, fc8cd5633bc64091b0ff7b294d79bcfb; no voters:
I20260812 05:16:06.955457 27543 raft_consensus.cc:2781] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:06.955557 27543 raft_consensus.cc:686] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 1 LEADER]: Becoming Leader. State: Replica: a06500afa15f4806b343abb20d1021fe, State: Running, Role: LEADER
I20260812 05:16:06.955612 27543 consensus_queue.cc:237] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } }
I20260812 05:16:06.956346 26995 catalog_manager.cc:5495] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe reported cstate change: term changed from 0 to 1, leader changed from <none> to a06500afa15f4806b343abb20d1021fe (127.18.166.195). New cstate: current_term: 1 leader_uuid: "a06500afa15f4806b343abb20d1021fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06500afa15f4806b343abb20d1021fe" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43059 } health_report { overall_health: HEALTHY } } }
I20260812 05:16:06.992774 27532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.990541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:38483 (local address 127.18.166.194:46739)
0812 05:16:06.990594 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.990596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.990727 (+ 131us) server_negotiation.cc:407] Connection header received
0812 05:16:06.990775 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.990775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.990785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:06.990887 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:06.990997 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.991493 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.992225 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.992422 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.992475 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.992609 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.992633 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.992666 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:06.992699 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 05:16:06.992714 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:16:06.992798 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.990481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:46739 (local address 127.18.166.195:38483)
0812 05:16:06.990649 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.990663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.990733 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:06.990885 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:06.990886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:06.990890 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:16:06.990941 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.990942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.991560 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.991561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.992142 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:06.992143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:06.992489 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:06.992489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:06.992529 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:06.992544 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:16:06.992667 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:16:06.992668 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:16:06.992697 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 05:16:06.992731 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260812 05:16:06.994800 27310 raft_consensus.cc:1261] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Refusing update from remote peer a06500afa15f4806b343abb20d1021fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.994802 27164 raft_consensus.cc:1261] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Refusing update from remote peer a06500afa15f4806b343abb20d1021fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:06.995036 27543 consensus_queue.cc:1035] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8cd5633bc64091b0ff7b294d79bcfb" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:06.995169 27613 consensus_queue.cc:1035] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6546b343d096402abf1916ec60b7aa6b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 46739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:07.002203 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.999800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43059 (local address 127.18.166.194:39671)
0812 05:16:06.999838 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:06.999850 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:06.999888 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:07.000036 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:07.000036 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:07.000037 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:16:07.000074 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:07.000075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:07.000786 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:07.000786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:07.001319 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:07.001321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:07.001746 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:07.001748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:07.001814 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:07.001823 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:16:07.002016 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:16:07.002019 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:07.002063 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 05:16:07.002116 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 05:16:07.002535 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:06.999888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:39671 (local address 127.18.166.195:43059)
0812 05:16:06.999933 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:06.999935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:06.999943 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:06.999963 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:06.999963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:06.999973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:07.000035 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:07.000143 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:07.000718 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:07.001389 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:07.001666 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:07.001733 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:07.001917 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:07.001951 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:07.002000 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:07.002443 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 05:16:07.002466 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:16:07.009218 26996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:07.009320 26996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:07.010010 26996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0440ae18699a40f58120dcb862bef859: Sending DeleteTablet for 3 replicas of tablet 9bad0356b8c5455e92bfcc1ae87b4f1e
I20260812 05:16:07.010259 27143 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:07 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44102
I20260812 05:16:07.010279 27434 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:07 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:55130
I20260812 05:16:07.010337 27290 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bad0356b8c5455e92bfcc1ae87b4f1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:07 UTC) from {username='oryx', principal='oryx/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59518
I20260812 05:16:07.010372 27559 tablet_replica.cc:330] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: stopping tablet replica
I20260812 05:16:07.010388 27558 tablet_replica.cc:330] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: stopping tablet replica
I20260812 05:16:07.010424 27557 tablet_replica.cc:330] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: stopping tablet replica
I20260812 05:16:07.010422 27559 raft_consensus.cc:2229] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:07.010444 27558 raft_consensus.cc:2229] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:07.010457 27559 raft_consensus.cc:2258] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:07.010478 27557 raft_consensus.cc:2229] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:07.010515 27557 raft_consensus.cc:2258] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:07.010525 27558 raft_consensus.cc:2258] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:07.010814 27559 ts_tablet_manager.cc:1905] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:07.010834 27557 ts_tablet_manager.cc:1905] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:07.010890 27558 ts_tablet_manager.cc:1905] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:07.012193 27558 ts_tablet_manager.cc:1918] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:16:07.012311 27558 log.cc:1196] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/9bad0356b8c5455e92bfcc1ae87b4f1e
I20260812 05:16:07.012347 27559 ts_tablet_manager.cc:1918] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:16:07.012418 27559 log.cc:1196] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/9bad0356b8c5455e92bfcc1ae87b4f1e
I20260812 05:16:07.012569 27558 ts_tablet_manager.cc:1939] T 9bad0356b8c5455e92bfcc1ae87b4f1e P a06500afa15f4806b343abb20d1021fe: Deleting consensus metadata
I20260812 05:16:07.012634 27559 ts_tablet_manager.cc:1939] T 9bad0356b8c5455e92bfcc1ae87b4f1e P fc8cd5633bc64091b0ff7b294d79bcfb: Deleting consensus metadata
I20260812 05:16:07.012970 26982 catalog_manager.cc:4841] TS a06500afa15f4806b343abb20d1021fe (127.18.166.195:43059): tablet 9bad0356b8c5455e92bfcc1ae87b4f1e (table test-table [id=e3086b4c54b2443c87ee8687cfcd97d4]) successfully deleted
I20260812 05:16:07.013056 26982 catalog_manager.cc:4841] TS fc8cd5633bc64091b0ff7b294d79bcfb (127.18.166.193:33963): tablet 9bad0356b8c5455e92bfcc1ae87b4f1e (table test-table [id=e3086b4c54b2443c87ee8687cfcd97d4]) successfully deleted
I20260812 05:16:07.013116 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 27072
I20260812 05:16:07.017470 27557 ts_tablet_manager.cc:1918] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:16:07.017561 27557 log.cc:1196] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/9bad0356b8c5455e92bfcc1ae87b4f1e
I20260812 05:16:07.017781 27557 ts_tablet_manager.cc:1939] T 9bad0356b8c5455e92bfcc1ae87b4f1e P 6546b343d096402abf1916ec60b7aa6b: Deleting consensus metadata
I20260812 05:16:07.018111 26983 catalog_manager.cc:4841] TS 6546b343d096402abf1916ec60b7aa6b (127.18.166.194:46739): tablet 9bad0356b8c5455e92bfcc1ae87b4f1e (table test-table [id=e3086b4c54b2443c87ee8687cfcd97d4]) successfully deleted
I20260812 05:16:07.022709 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 27224
I20260812 05:16:07.029894 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 27368
I20260812 05:16:07.036684 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 26965
2026-08-12T05:16:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2845 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:16:07 dist-test-slave-srgv krb5kdc[27631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:16:07 dist-test-slave-srgv krb5kdc[27631](info): set up 2 sockets
Aug 12 05:16:07 dist-test-slave-srgv krb5kdc[27631](info): commencing operation
krb5kdc: starting...
W20260812 05:16:08.120018 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:16:08 dist-test-slave-srgv krb5kdc[27631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:16:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:16:08.255069 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39277
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:39277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:08.343717 27644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:08.343983 27644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:08.344012 27644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:08.345343 27644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:08.345440 27644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:08.345458 27644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:08.345471 27644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:08.345484 27644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:08.347169 27644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:39277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:08.347415 27644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:08.347615 27644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:08.353061 27649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:08.353091 27653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:08.353061 27650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:08.353524 27644 server_base.cc:1034] running on GCE node
I20260812 05:16:08.353837 27644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:08.354178 27644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:08.355373 27644 hybrid_clock.cc:648] HybridClock initialized: now 1786511768355357 us; error 53 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T05:16:08Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1326 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:16:08 dist-test-slave-srgv krb5kdc[27658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:16:08 dist-test-slave-srgv krb5kdc[27658](info): set up 2 sockets
Aug 12 05:16:08 dist-test-slave-srgv krb5kdc[27658](info): commencing operation
krb5kdc: starting...
W20260812 05:16:09.448869 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:16:09 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:16:09.584764 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35987
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:09.675321 27672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:09.675655 27672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:09.675705 27672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:09.677134 27672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:09.677245 27672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:09.677273 27672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:09.677297 27672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:09.677328 27672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:09.679217 27672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:09.679580 27672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:09.679852 27672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:09.685283 27677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:09.685322 27678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:09.685283 27680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:09.685572 27672 server_base.cc:1034] running on GCE node
I20260812 05:16:09.685871 27672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:09.686223 27672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:09.687433 27672 hybrid_clock.cc:648] HybridClock initialized: now 1786511769687416 us; error 67 us; skew 500 ppm
Aug 12 05:16:09 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511769, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:09.697139 27672 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:16:09.697788 27672 webserver.cc:460] Webserver started at http://127.18.166.254:36081/ using document root <none> and password file <none>
I20260812 05:16:09.698062 27672 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:09.698148 27672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:09.698318 27672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:09.699570 27672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "90dc442f1f314b81af3d238510b1a99a"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "3fdaef59b326e8dae4cd367ff7aca8d7"
server_key_iv: "bceb2e600d14a28338b4c99763dbeab0"
server_key_version: "encryptionkey@0"
I20260812 05:16:09.699999 27672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "90dc442f1f314b81af3d238510b1a99a"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "3fdaef59b326e8dae4cd367ff7aca8d7"
server_key_iv: "bceb2e600d14a28338b4c99763dbeab0"
server_key_version: "encryptionkey@0"
I20260812 05:16:09.701649 27672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:16:09.702493 27688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:09.702703 27672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:09.702823 27672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "90dc442f1f314b81af3d238510b1a99a"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "3fdaef59b326e8dae4cd367ff7aca8d7"
server_key_iv: "bceb2e600d14a28338b4c99763dbeab0"
server_key_version: "encryptionkey@0"
I20260812 05:16:09.702906 27672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:09.724542 27672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:09.726759 27672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:09.726948 27672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:09.731734 27672 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:35987
I20260812 05:16:09.731794 27744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:35987 every 8 connection(s)
I20260812 05:16:09.732214 27672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:09.733172 27745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:09.735852 27745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Bootstrap starting.
I20260812 05:16:09.736495 27745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:09.736773 27745 log.cc:824] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:09.737771 27745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: No bootstrap required, opened a new log
I20260812 05:16:09.739513 27745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:09.739688 27745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:09.739713 27745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90dc442f1f314b81af3d238510b1a99a, State: Initialized, Role: FOLLOWER
I20260812 05:16:09.739876 27745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:09.739970 27745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:09.739996 27745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:09.740028 27745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:09.740653 27745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:09.740756 27745 leader_election.cc:302] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90dc442f1f314b81af3d238510b1a99a; no voters:
I20260812 05:16:09.740978 27745 leader_election.cc:288] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:16:09.741022 27748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:09.741154 27748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 LEADER]: Becoming Leader. State: Replica: 90dc442f1f314b81af3d238510b1a99a, State: Running, Role: LEADER
I20260812 05:16:09.741291 27745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:09.741343 27748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:09.741688 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 27672
I20260812 05:16:09.741822 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:16:09.742069 19099 external_mini_cluster.cc:1428] Setting key 15f0c573990cc2f0cee71c55dd8682fd
I20260812 05:16:09.742792 27750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90dc442f1f314b81af3d238510b1a99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } } }
I20260812 05:16:09.742992 27749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90dc442f1f314b81af3d238510b1a99a. Latest consensus state: current_term: 1 leader_uuid: "90dc442f1f314b81af3d238510b1a99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } } }
I20260812 05:16:09.743104 27750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:09.743116 27749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:09.743525 27767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:09.744112 27767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:16:09.745807 27767 catalog_manager.cc:1347] Generated new cluster ID: 67e56fadc53347d0b9ba121192367867
I20260812 05:16:09.745869 27767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:16:09 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:09.753023 27767 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:16:09.753808 27767 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:09.754597 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:09.742645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.18.166.254:35987)
0812 05:16:09.743257 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:09.743268 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:09.743522 (+ 254us) server_negotiation.cc:407] Connection header received
0812 05:16:09.743657 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:09.743666 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:09.743729 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:09.743897 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:09.744807 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:09.746145 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:09.746954 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:09.747392 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:09.747519 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:09.750091 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:09.750095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:09.750096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:09.750110 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:09.752642 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:09.753232 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:09.753235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:09.753236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:09.753347 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:09.753621 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:09.753622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:09.753626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:09.753906 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:09.754141 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:09.754231 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 05:16:09.754403 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":81,"threads_started":1}
I20260812 05:16:09.761534 27767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Generated new TSK 0
I20260812 05:16:09.761793 27767 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:16:09.793571 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35987
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:09.885926 27773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:09.886178 27773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:09.886209 27773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:09.887543 27773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:09.887602 27773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:09.887631 27773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:09.889362 27773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:09.889672 27773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:09.889950 27773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:09.895444 27779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:09.895558 27773 server_base.cc:1034] running on GCE node
W20260812 05:16:09.895448 27778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:09.895448 27782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:09.895831 27773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:09.896126 27773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:09.897303 27773 hybrid_clock.cc:648] HybridClock initialized: now 1786511769897299 us; error 40 us; skew 500 ppm
Aug 12 05:16:09 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511769, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:09.900146 27773 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:16:09.900648 27773 webserver.cc:460] Webserver started at http://127.18.166.193:35717/ using document root <none> and password file <none>
I20260812 05:16:09.900873 27773 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:09.900918 27773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:09.901015 27773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:09.902150 27773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "18ead767bfaa4aed8476bde1d2b83c87"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "9d8de8cba7b748c857cd210d19d0b54a"
server_key_iv: "1baaacd233786c8b74b4914053bcbf45"
server_key_version: "encryptionkey@0"
I20260812 05:16:09.902496 27773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "18ead767bfaa4aed8476bde1d2b83c87"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "9d8de8cba7b748c857cd210d19d0b54a"
server_key_iv: "1baaacd233786c8b74b4914053bcbf45"
server_key_version: "encryptionkey@0"
I20260812 05:16:09.903923 27773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:16:09.904713 27789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:09.904930 27773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:09.905030 27773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "18ead767bfaa4aed8476bde1d2b83c87"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "9d8de8cba7b748c857cd210d19d0b54a"
server_key_iv: "1baaacd233786c8b74b4914053bcbf45"
server_key_version: "encryptionkey@0"
I20260812 05:16:09.905094 27773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:09.912752 27773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:09.915145 27773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:09.915283 27773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:09.915503 27773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:09.915896 27773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:09.915936 27773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:09.915989 27773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:09.916015 27773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:09.922546 27773 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:43823
I20260812 05:16:09.922631 27913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:43823 every 8 connection(s)
I20260812 05:16:09.923012 27773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:09.929756 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 27773
I20260812 05:16:09.929891 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
Aug 12 05:16:09 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511769, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:09.930173 19099 external_mini_cluster.cc:1428] Setting key b7a7c2e18d9d62e27de70b2733fa9f60
I20260812 05:16:09.934187 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:09.923934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:51529 (local address 127.18.166.254:35987)
0812 05:16:09.924029 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:09.924032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:09.924247 (+ 215us) server_negotiation.cc:407] Connection header received
0812 05:16:09.924823 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:09.924825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:09.924838 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:09.924889 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:09.926015 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:09.926536 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:09.927585 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:09.927805 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:09.927900 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:09.930596 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:09.930598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:09.930599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:09.930609 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:09.932296 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:09.932936 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:09.932938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:09.932938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:09.933016 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:09.933305 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:09.933306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:09.933307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:09.933528 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:09.933620 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:09.933949 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 05:16:09.934063 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:16:09.934386 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:09.923813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:51529)
0812 05:16:09.924076 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:09.924147 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:09.924696 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:09.924968 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:09.924971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:09.925680 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:09.925897 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:09.925902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:09.926654 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:09.926655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:09.927478 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:09.927480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:09.927918 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:09.927919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:09.928087 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:09.928700 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:09.928709 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:09.930438 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:09.932405 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:09.932415 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:09.932419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:09.932831 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:09.933101 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:09.933102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:09.933102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:09.933233 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:09.933620 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:09.933629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:09.933786 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 05:16:09.933894 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 05:16:09.934097 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":117,"threads_started":1}
I20260812 05:16:09.934975 27915 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35987
I20260812 05:16:09.935108 27915 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:09.935324 27915 heartbeater.cc:511] Master 127.18.166.254:35987 requested a full tablet report, sending...
I20260812 05:16:09.936093 27706 ts_manager.cc:194] Registered new tserver with Master: 18ead767bfaa4aed8476bde1d2b83c87 (127.18.166.193:43823)
I20260812 05:16:09.937130 27706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:51529
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:16:09.960816 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35987
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:10.048800 27920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:10.049059 27920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:10.049088 27920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:10.050637 27920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:10.050719 27920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:10.050755 27920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:10.052452 27920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:10.052692 27920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:10.052899 27920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:10.058380 27929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.058400 27926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.058398 27925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.058566 27920 server_base.cc:1034] running on GCE node
I20260812 05:16:10.058878 27920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:10.059151 27920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:10.060324 27920 hybrid_clock.cc:648] HybridClock initialized: now 1786511770060309 us; error 38 us; skew 500 ppm
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:10.066134 27920 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:16:10.066648 27920 webserver.cc:460] Webserver started at http://127.18.166.194:42667/ using document root <none> and password file <none>
I20260812 05:16:10.066861 27920 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:10.066906 27920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:10.067011 27920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:10.068221 27920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "f3dae386339a4c709e0d13a3adefcacc"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "0f1f6059a82f386e6931e918486f1adc"
server_key_iv: "73610ffedbe698a76bd27dd06dedd184"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.068567 27920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "f3dae386339a4c709e0d13a3adefcacc"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "0f1f6059a82f386e6931e918486f1adc"
server_key_iv: "73610ffedbe698a76bd27dd06dedd184"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.070137 27920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:16:10.071080 27936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.071370 27920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:10.071481 27920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "f3dae386339a4c709e0d13a3adefcacc"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "0f1f6059a82f386e6931e918486f1adc"
server_key_iv: "73610ffedbe698a76bd27dd06dedd184"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.071569 27920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:10.092569 27920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:10.095309 27920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:10.095498 27920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:10.095777 27920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:10.096136 27920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:10.096174 27920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.096199 27920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:10.096241 27920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.102942 27920 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:42331
I20260812 05:16:10.103022 28062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:42331 every 8 connection(s)
I20260812 05:16:10.103371 27920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:10.108021 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 27920
I20260812 05:16:10.108199 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:16:10.108402 19099 external_mini_cluster.cc:1428] Setting key 25354a7382051244431bc332624530f6
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:10.115192 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.104333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:41675 (local address 127.18.166.254:35987)
0812 05:16:10.104425 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.104428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.104645 (+ 217us) server_negotiation.cc:407] Connection header received
0812 05:16:10.105103 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.105105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.105117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.105198 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.106359 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.106867 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.108007 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.108395 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.108496 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.111156 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:10.111158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:10.111159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:10.111170 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:10.113148 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:10.113802 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:10.113804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:10.113804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:10.113884 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:10.114320 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:10.114321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:10.114322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:10.114508 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:10.114592 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:10.114912 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 05:16:10.115026 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 05:16:10.115262 28066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.104273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:41675)
0812 05:16:10.104462 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.104508 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.105013 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.105258 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.105261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.106018 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.106243 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.106249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.106993 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.106994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.107884 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.107886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.108516 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.108517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.108657 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.109335 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.109350 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.111016 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:10.113285 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:10.113297 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:10.113300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:10.113703 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:10.114056 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:10.114056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:10.114057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:10.114220 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:10.114588 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:10.114594 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:10.114742 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 05:16:10.114829 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:16:10.114985 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260812 05:16:10.115926 28063 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35987
I20260812 05:16:10.116060 28063 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:10.116310 28063 heartbeater.cc:511] Master 127.18.166.254:35987 requested a full tablet report, sending...
I20260812 05:16:10.116815 27706 ts_manager.cc:194] Registered new tserver with Master: f3dae386339a4c709e0d13a3adefcacc (127.18.166.194:42331)
I20260812 05:16:10.117271 27706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:41675
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:16:10.140780 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35987
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:10.229439 28070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:10.229817 28070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:10.229862 28070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:10.231398 28070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:10.231503 28070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:10.231554 28070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:10.233230 28070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:35987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:10.233575 28070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:10.233865 28070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:10.239264 28076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.239264 28078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.239302 28080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.239580 28070 server_base.cc:1034] running on GCE node
I20260812 05:16:10.239969 28070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:10.240265 28070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:10.241493 28070 hybrid_clock.cc:648] HybridClock initialized: now 1786511770241438 us; error 68 us; skew 500 ppm
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:10.244313 28070 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:16:10.244864 28070 webserver.cc:460] Webserver started at http://127.18.166.195:43609/ using document root <none> and password file <none>
I20260812 05:16:10.245142 28070 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:10.245214 28070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:10.245371 28070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:10.246729 28070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "8a429152c05844ddb391a2eb62fe6199"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "3732ff1408f5a1a1cb9732e0397724df"
server_key_iv: "194a45559ab5cd0efad551604f313bbc"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.247298 28070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "8a429152c05844ddb391a2eb62fe6199"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "3732ff1408f5a1a1cb9732e0397724df"
server_key_iv: "194a45559ab5cd0efad551604f313bbc"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.249483 28070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:16:10.250700 28087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.250914 28070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:10.251027 28070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "8a429152c05844ddb391a2eb62fe6199"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "3732ff1408f5a1a1cb9732e0397724df"
server_key_iv: "194a45559ab5cd0efad551604f313bbc"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.251133 28070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:10.274994 28070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:10.278451 28070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:10.278654 28070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:10.278936 28070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:10.279373 28070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:10.279424 28070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.279448 28070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:10.279461 28070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.288735 28070 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43781
I20260812 05:16:10.288807 28208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43781 every 8 connection(s)
I20260812 05:16:10.289343 28070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:10.297338 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28070
I20260812 05:16:10.297510 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:16:10.297708 19099 external_mini_cluster.cc:1428] Setting key 1d18d53e22df8b8be1bd18ca135d0ef5
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:10.302996 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.290321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39047 (local address 127.18.166.254:35987)
0812 05:16:10.290391 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.290393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.290731 (+ 338us) server_negotiation.cc:407] Connection header received
0812 05:16:10.291577 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.291578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.291591 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.291670 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.293279 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.293886 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.295680 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.296025 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.296116 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.299668 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:10.299670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:10.299670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:10.299678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:10.301573 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:10.302065 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:10.302065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:10.302066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:10.302139 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:10.302375 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:10.302376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:10.302376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:10.302501 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:10.302556 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:10.302841 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 05:16:10.302922 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:16:10.303238 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.290233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:39047)
0812 05:16:10.290505 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.290570 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.291423 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.291773 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.291778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.292779 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.293138 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.293148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.293984 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.293986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.295523 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.295527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.296148 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.296150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.296335 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.297091 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.297105 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.299505 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:10.301680 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:10.301691 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:10.301694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:10.301981 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:10.302203 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:10.302204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:10.302204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:10.302315 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:10.302546 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:10.302549 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:10.302707 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 05:16:10.302806 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 05:16:10.302958 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":118,"threads_started":1}
I20260812 05:16:10.303823 28209 heartbeater.cc:347] Connected to a master server at 127.18.166.254:35987
I20260812 05:16:10.303951 28209 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:10.304171 28209 heartbeater.cc:511] Master 127.18.166.254:35987 requested a full tablet report, sending...
I20260812 05:16:10.304652 27706 ts_manager.cc:194] Registered new tserver with Master: 8a429152c05844ddb391a2eb62fe6199 (127.18.166.195:43781)
I20260812 05:16:10.305056 27706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:39047
I20260812 05:16:10.309690 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab.
Entry for principal oryx/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab.
WARNING: no policy specified for oryx/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab.
Entry for principal oryx/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab.
WARNING: no policy specified for oryx/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab.
Entry for principal oryx/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab.
I20260812 05:16:10.392364 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 27773
I20260812 05:16:10.400403 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 27920
I20260812 05:16:10.406924 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28070
I20260812 05:16:10.414280 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 27672
I20260812 05:16:10.421674 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35987
--webserver_interface=127.18.166.254
--webserver_port=36081
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:10.508615 28218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:10.508917 28218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:10.508965 28218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:10.510315 28218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:10.510412 28218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:10.510438 28218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:10.510459 28218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:10.510493 28218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:10.512154 28218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:35987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=36081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:10.512485 28218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:10.512763 28218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:10.518221 28224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.518246 28225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.518215 28227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.518251 28218 server_base.cc:1034] running on GCE node
I20260812 05:16:10.518811 28218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:10.519132 28218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:10.520311 28218 hybrid_clock.cc:648] HybridClock initialized: now 1786511770520301 us; error 53 us; skew 500 ppm
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:10.523082 28218 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:16:10.523576 28218 webserver.cc:460] Webserver started at http://127.18.166.254:36081/ using document root <none> and password file <none>
I20260812 05:16:10.523784 28218 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:10.523841 28218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:10.525225 28218 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:10.525997 28234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.526238 28218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:10.526314 28218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "90dc442f1f314b81af3d238510b1a99a"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "3fdaef59b326e8dae4cd367ff7aca8d7"
server_key_iv: "bceb2e600d14a28338b4c99763dbeab0"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.526654 28218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:10.554803 28218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:10.556936 28218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:10.557070 28218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:10.561715 28218 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:35987
I20260812 05:16:10.561751 28289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:35987 every 8 connection(s)
I20260812 05:16:10.562192 28218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:10.563862 28290 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:16:10.564646 28290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Bootstrap starting.
I20260812 05:16:10.568265 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28218
I20260812 05:16:10.568696 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:43823
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:35987
--webserver_port=35717
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:16:10.575744 28290 log.cc:824] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:10.577962 28290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:16:10.578346 28290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Bootstrap complete.
I20260812 05:16:10.581835 28290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:10.582175 28290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90dc442f1f314b81af3d238510b1a99a, State: Initialized, Role: FOLLOWER
I20260812 05:16:10.582397 28290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:10.582489 28290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:10.582553 28290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:10.582598 28290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:16:10.584115 28290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:10.584229 28290 leader_election.cc:302] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90dc442f1f314b81af3d238510b1a99a; no voters:
I20260812 05:16:10.584800 28290 leader_election.cc:288] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:16:10.584854 28294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:16:10.585106 28294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [term 2 LEADER]: Becoming Leader. State: Replica: 90dc442f1f314b81af3d238510b1a99a, State: Running, Role: LEADER
I20260812 05:16:10.585256 28294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } }
I20260812 05:16:10.585368 28290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:10.585805 28295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90dc442f1f314b81af3d238510b1a99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } } }
I20260812 05:16:10.586005 28295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:10.585700 28296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90dc442f1f314b81af3d238510b1a99a. Latest consensus state: current_term: 2 leader_uuid: "90dc442f1f314b81af3d238510b1a99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc442f1f314b81af3d238510b1a99a" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 35987 } } }
I20260812 05:16:10.586355 28296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:10.586452 28304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:10.587111 28304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:16:10.587288 28304 catalog_manager.cc:1259] Loaded cluster ID: 67e56fadc53347d0b9ba121192367867
I20260812 05:16:10.587597 28304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:10.588501 28304 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:10.588739 28304 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90dc442f1f314b81af3d238510b1a99a: Loaded TSK: 0
I20260812 05:16:10.588982 28304 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:16:10.677335 28292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:10.677716 28292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:10.677765 28292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:10.679191 28292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:10.679283 28292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:10.679334 28292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:10.681051 28292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.193.keytab
--principal=oryx/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:43823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=35717
--tserver_master_addrs=127.18.166.254:35987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:10.681389 28292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:10.681684 28292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:10.687243 28317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.687247 28319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.687247 28316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.688299 28292 server_base.cc:1034] running on GCE node
I20260812 05:16:10.688566 28292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:10.688866 28292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:10.690049 28292 hybrid_clock.cc:648] HybridClock initialized: now 1786511770690029 us; error 36 us; skew 500 ppm
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:10.693323 28292 init.cc:377] Logged in from keytab as oryx/127.18.166.193@KRBTEST.COM (short username oryx)
I20260812 05:16:10.693980 28292 webserver.cc:460] Webserver started at http://127.18.166.193:35717/ using document root <none> and password file <none>
I20260812 05:16:10.694243 28292 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:10.694347 28292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:10.695926 28292 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:16:10.696687 28326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.696907 28292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:16:10.697024 28292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "18ead767bfaa4aed8476bde1d2b83c87"
format_stamp: "Formatted at 2026-08-12 05:16:09 on dist-test-slave-srgv"
server_key: "9d8de8cba7b748c857cd210d19d0b54a"
server_key_iv: "1baaacd233786c8b74b4914053bcbf45"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.697470 28292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:10.710497 28292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:10.713131 28292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:10.713323 28292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:10.713639 28292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:10.714027 28292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:10.714087 28292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.714151 28292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:10.714172 28292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.721664 28292 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:43823
I20260812 05:16:10.721714 28452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:43823 every 8 connection(s)
I20260812 05:16:10.722139 28292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:10.726095 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28292
I20260812 05:16:10.726459 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:42331
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:35987
--webserver_port=42667
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:10.732977 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.723055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:46817)
0812 05:16:10.723269 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.723329 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.723996 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.724504 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.724508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.725170 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.725376 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.725381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.726848 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.726850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.727682 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.727685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.728745 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.728746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.728911 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.729646 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.729658 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.732713 (+ 3055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
W20260812 05:16:10.733114 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:10.723380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:46817 (local address 127.18.166.254:35987)
0812 05:16:10.723606 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.723615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.723660 (+ 45us) server_negotiation.cc:407] Connection header received
0812 05:16:10.724266 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.724271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.724321 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.724463 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.725547 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.726735 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.727789 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.728507 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.728631 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.732907 (+ 4276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:46817: BlockingRecv error: recv got EOF from 127.18.166.193:46817 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
W20260812 05:16:10.733417 28453 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:35987 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.166.254:35987: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:10.740252 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.733742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:54931)
0812 05:16:10.733877 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.733888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.733933 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.734153 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.734154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.734347 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.734426 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.734428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.735395 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.735397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.736371 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.736375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.736841 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.736843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.736929 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.737513 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.737519 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.740163 (+ 2644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260812 05:16:10.740792 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:10.733841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:54931 (local address 127.18.166.254:35987)
0812 05:16:10.734005 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.734008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.734015 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:16:10.734042 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.734043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.734055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.734108 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.734538 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.735288 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.736487 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.736747 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.736824 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.740724 (+ 3900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:54931: BlockingRecv error: recv got EOF from 127.18.166.193:54931 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:10.747231 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.740890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:58059)
0812 05:16:10.741140 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.741152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.741209 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.741496 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.741497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.741671 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.741721 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.741722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.742694 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.742695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.743538 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.743540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.744070 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.744071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.744134 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.744691 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.744696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.747159 (+ 2463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231}
W20260812 05:16:10.747414 28453 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:16:10.747704 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:10.740992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:58059 (local address 127.18.166.254:35987)
0812 05:16:10.741030 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.741032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.741195 (+ 163us) server_negotiation.cc:407] Connection header received
0812 05:16:10.741380 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.741381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.741395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.741489 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.741800 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.742593 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.743662 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.743979 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.744047 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.747625 (+ 3578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:58059: BlockingRecv error: recv got EOF from 127.18.166.193:58059 (error 108)
Metrics: {"server-negotiator.queue_time_us":19,"thread_start_us":108,"threads_started":1}
W20260812 05:16:10.818064 28457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:10.818310 28457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:10.818339 28457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:10.819670 28457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:10.819731 28457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:10.819764 28457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:10.821533 28457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.194.keytab
--principal=oryx/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:42331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=42667
--tserver_master_addrs=127.18.166.254:35987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:10.821828 28457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:10.822108 28457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:10.827548 28464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.827548 28467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.827764 28457 server_base.cc:1034] running on GCE node
W20260812 05:16:10.827589 28463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.828071 28457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:10.828394 28457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:10.829555 28457 hybrid_clock.cc:648] HybridClock initialized: now 1786511770829542 us; error 36 us; skew 500 ppm
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:10.832657 28457 init.cc:377] Logged in from keytab as oryx/127.18.166.194@KRBTEST.COM (short username oryx)
I20260812 05:16:10.833232 28457 webserver.cc:460] Webserver started at http://127.18.166.194:42667/ using document root <none> and password file <none>
I20260812 05:16:10.833575 28457 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:10.833673 28457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:10.835207 28457 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:10.835863 28474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.836054 28457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:10.836179 28457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "f3dae386339a4c709e0d13a3adefcacc"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "0f1f6059a82f386e6931e918486f1adc"
server_key_iv: "73610ffedbe698a76bd27dd06dedd184"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.836565 28457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:10.868373 28457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:10.870985 28457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:10.871183 28457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:10.871457 28457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:10.871797 28457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:10.871855 28457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.871897 28457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:10.871941 28457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.878118 28457 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:42331
I20260812 05:16:10.878165 28591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:42331 every 8 connection(s)
I20260812 05:16:10.878548 28457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:10.883965 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28457
I20260812 05:16:10.884326 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:43781
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:35987
--webserver_port=43609
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:10.886701 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.879377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:45553)
0812 05:16:10.879567 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.879607 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.880117 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.880373 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.880378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.881008 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.881223 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.881228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.881918 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.881918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.882758 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.882761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.883130 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.883131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.883258 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.883780 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.883789 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.886528 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
W20260812 05:16:10.886747 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:10.879377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:45553 (local address 127.18.166.254:35987)
0812 05:16:10.879446 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.879448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.879688 (+ 240us) server_negotiation.cc:407] Connection header received
0812 05:16:10.880218 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.880219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.880234 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.880291 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.881313 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.881832 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.882843 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.883031 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.883117 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.886663 (+ 3546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:45553: BlockingRecv error: recv got EOF from 127.18.166.194:45553 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 05:16:10.886976 28592 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:35987 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.166.254:35987: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:10.892520 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.887218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:35363)
0812 05:16:10.887269 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.887282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.887321 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.887498 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.887498 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.887655 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.887715 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.887717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.888403 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.888405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.889316 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.889319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.889728 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.889729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.889812 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.890259 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.890266 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.892430 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 05:16:10.892575 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:10.887306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:35363 (local address 127.18.166.254:35987)
0812 05:16:10.887370 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.887372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.887381 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:10.887411 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.887412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.887420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.887470 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.887817 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.888312 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.889464 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.889646 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.889705 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.892511 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:35363: BlockingRecv error: recv got EOF from 127.18.166.194:35363 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:10.897716 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:10.892848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:33297)
0812 05:16:10.892895 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:10.892906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:10.892944 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:10.893098 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:10.893099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:10.893272 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:10.893315 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.893317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.893982 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.893983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.894864 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:10.894867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.895240 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.895241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:10.895305 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.895718 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:10.895724 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:10.897664 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 05:16:10.897879 28592 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:16:10.897902 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:10.892915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:33297 (local address 127.18.166.254:35987)
0812 05:16:10.892967 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:10.892969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:10.892979 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:10.893008 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:10.893009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:10.893016 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:10.893073 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:10.893392 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.893893 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.894964 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:10.895157 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:10.895215 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:10.897771 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:33297: BlockingRecv error: recv got EOF from 127.18.166.194:33297 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 05:16:10.972177 28595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:10.972429 28595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:10.972457 28595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:10.973892 28595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:10.973958 28595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:10.973992 28595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:10.975627 28595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/krb5kdc/oryx_127.18.166.195.keytab
--principal=oryx/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:43781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=43609
--tserver_master_addrs=127.18.166.254:35987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:10.975890 28595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:10.976094 28595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:10.981362 28604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.981395 28602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:10.981362 28601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:10.981571 28595 server_base.cc:1034] running on GCE node
I20260812 05:16:10.981935 28595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:10.982220 28595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:10.983422 28595 hybrid_clock.cc:648] HybridClock initialized: now 1786511770983417 us; error 41 us; skew 500 ppm
Aug 12 05:16:10 dist-test-slave-srgv krb5kdc[27658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511770, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:10.986469 28595 init.cc:377] Logged in from keytab as oryx/127.18.166.195@KRBTEST.COM (short username oryx)
I20260812 05:16:10.987054 28595 webserver.cc:460] Webserver started at http://127.18.166.195:43609/ using document root <none> and password file <none>
I20260812 05:16:10.987329 28595 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:10.987426 28595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:10.989238 28595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:10.990314 28611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:10.990653 28595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:10.990806 28595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "8a429152c05844ddb391a2eb62fe6199"
format_stamp: "Formatted at 2026-08-12 05:16:10 on dist-test-slave-srgv"
server_key: "3732ff1408f5a1a1cb9732e0397724df"
server_key_iv: "194a45559ab5cd0efad551604f313bbc"
server_key_version: "encryptionkey@0"
I20260812 05:16:10.991245 28595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:11.007372 28595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:11.009626 28595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:11.009819 28595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:11.010102 28595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:11.010455 28595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:11.010516 28595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:11.010563 28595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:11.010589 28595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:11.017120 28595 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43781
I20260812 05:16:11.017168 28729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43781 every 8 connection(s)
I20260812 05:16:11.017647 28595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:11.021258 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28595
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 05:16:11.026639 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:11.018498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:46535 (local address 127.18.166.254:35987)
0812 05:16:11.018580 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:11.018583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:11.018813 (+ 230us) server_negotiation.cc:407] Connection header received
0812 05:16:11.019426 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:11.019427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:11.019441 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:11.019489 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:11.020545 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.021055 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.022414 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.022603 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.022653 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.026552 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:46535: BlockingRecv error: recv got EOF from 127.18.166.195:46535 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:16:11.026736 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:11.018495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:46535)
0812 05:16:11.018662 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:11.018711 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:11.019282 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:11.019579 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:11.019584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:11.020213 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:11.020414 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.020419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.021172 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.021173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.022333 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.022336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.022694 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.022695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.022882 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.023521 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:11.023530 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:11.026469 (+ 2939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
W20260812 05:16:11.027189 28730 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:35987 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.166.254:35987: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
I20260812 05:16:11.030318 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:11.021913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.18.166.254:35987)
0812 05:16:11.021980 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:11.021982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:11.022043 (+ 61us) server_negotiation.cc:407] Connection header received
0812 05:16:11.022130 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:11.022131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:11.022141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:11.022204 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:11.023117 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.023585 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.024700 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.024984 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.025044 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.026684 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:11.026689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:11.026690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:11.026704 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:11.028664 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:11.029073 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:11.029076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:11.029076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:11.029158 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:11.029393 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:11.029394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:11.029395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:11.029760 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:11.029898 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:11.030098 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 05:16:11.030224 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:11.032331 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:11.027428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:48625)
0812 05:16:11.027487 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:11.027499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:11.027542 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:11.027647 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:11.027647 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:11.027785 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:11.027839 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.027841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.028496 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.028497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.029258 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.029261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.029711 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.029712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.029769 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.030270 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:11.030284 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:11.032256 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 05:16:11.032388 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:11.027493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:48625 (local address 127.18.166.254:35987)
0812 05:16:11.027539 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:11.027541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:11.027550 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:11.027589 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:11.027590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:11.027599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:11.027645 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:11.027917 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.028410 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.029358 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.029643 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.029711 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.032346 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:48625: BlockingRecv error: recv got EOF from 127.18.166.195:48625 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:11.037245 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:11.032653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:56195)
0812 05:16:11.032707 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:11.032718 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:11.032763 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:11.032892 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:11.032893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:11.033031 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:11.033064 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.033065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.033898 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.033899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.034407 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.034409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.034806 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.034807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.034866 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.035364 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:11.035370 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:11.037150 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:16:11.037297 28456 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:11.032713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:56195 (local address 127.18.166.254:35987)
0812 05:16:11.032759 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:11.032761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:11.032770 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:11.032821 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:11.032822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:11.032832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:11.032887 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:11.033143 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.033811 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.034500 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.034712 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.034805 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.037237 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:56195: BlockingRecv error: recv got EOF from 127.18.166.195:56195 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:16:11.037498 28730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:11.753173 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:11.747959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:35281)
0812 05:16:11.748136 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:11.748156 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:11.748211 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:11.748411 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:11.748412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:11.748595 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:11.748648 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.748650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.749322 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.749324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.749925 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.749927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.750303 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.750304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.750358 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.750891 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:11.750896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:11.753044 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
W20260812 05:16:11.753336 28735 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:11.748105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:35281 (local address 127.18.166.254:35987)
0812 05:16:11.748260 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:11.748262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:11.748272 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:11.748301 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:11.748302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:11.748312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:11.748392 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:11.748722 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.749234 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.750013 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.750225 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.750287 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.753183 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:35281: BlockingRecv error: recv got EOF from 127.18.166.193:35281 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:11 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:11.903723 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:11.898251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:38103)
0812 05:16:11.898441 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:11.898454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:11.898558 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:11.898729 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:11.898731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:11.898938 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:11.898991 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.898993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.899718 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.899720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.900407 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:11.900410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.900786 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.900787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:11.900854 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.901465 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:11.901477 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:11.903627 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
W20260812 05:16:11.903829 28735 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:11.898340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:38103 (local address 127.18.166.254:35987)
0812 05:16:11.898422 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:11.898425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:11.898535 (+ 110us) server_negotiation.cc:407] Connection header received
0812 05:16:11.898631 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:11.898632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:11.898647 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:11.898722 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:11.899080 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.899616 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.900502 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:11.900705 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:11.900788 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:11.903762 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:38103: BlockingRecv error: recv got EOF from 127.18.166.194:38103 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 05:16:12 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:12 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:12.043453 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:12.037971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:33571)
0812 05:16:12.038162 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:12.038178 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:12.038293 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:12.038474 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:12.038475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:12.038675 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:12.038732 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:12.038734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.039438 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.039439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:12.040071 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:12.040075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.040512 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.040513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:12.040572 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:12.041237 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:12.041245 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:12.043379 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
W20260812 05:16:12.043637 28735 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:12.038096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:33571 (local address 127.18.166.254:35987)
0812 05:16:12.038181 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:12.038192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:12.038269 (+ 77us) server_negotiation.cc:407] Connection header received
0812 05:16:12.038373 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:12.038374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:12.038389 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:12.038465 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:12.038816 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.039332 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.040226 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.040428 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.040513 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:12.043567 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:33571: BlockingRecv error: recv got EOF from 127.18.166.195:33571 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 05:16:12 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:12 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:12.758991 28739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:12.753797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:36933)
0812 05:16:12.753966 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:12.753979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:12.754040 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:12.754253 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:12.754253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:12.754430 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:12.754483 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:12.754485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.755180 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.755181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:12.755710 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:12.755712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.756080 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.756081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:12.756124 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:12.756629 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:12.756634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:12.758925 (+ 2291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
W20260812 05:16:12.759136 28740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:12.753911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:36933 (local address 127.18.166.254:35987)
0812 05:16:12.754079 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:12.754081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:12.754098 (+ 17us) server_negotiation.cc:407] Connection header received
0812 05:16:12.754137 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:12.754137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:12.754148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:12.754228 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:12.754561 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.755088 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.755798 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.755997 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.756083 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:12.759039 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:36933: BlockingRecv error: recv got EOF from 127.18.166.193:36933 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
Aug 12 05:16:12 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:12 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:12.910091 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:12.904423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:49249)
0812 05:16:12.904601 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:12.904615 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:12.904678 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:12.904799 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:12.904799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:12.904982 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:12.905035 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:12.905037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.905818 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.905819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:12.906482 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:12.906486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.907030 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.907031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:12.907084 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:12.907745 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:12.907751 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:12.910025 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
W20260812 05:16:12.910298 28740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:12.904613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:49249 (local address 127.18.166.254:35987)
0812 05:16:12.904679 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:12.904681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:12.904715 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:16:12.904740 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:12.904741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:12.904752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:12.904795 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:12.905113 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.905718 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.906624 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:12.906833 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:12.906904 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:12.910209 (+ 3305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:49249: BlockingRecv error: recv got EOF from 127.18.166.194:49249 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 05:16:13 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:13 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:13.049865 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:13.044209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:39237)
0812 05:16:13.044424 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:13.044438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:13.044527 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:13.044692 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:13.044693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:13.044880 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:13.044935 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:13.044936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.045930 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.045931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:13.046671 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:13.046674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.047051 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.047052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:13.047094 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:13.047586 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:13.047591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:13.049768 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":87,"threads_started":1}
W20260812 05:16:13.049995 28740 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:13.044415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39237 (local address 127.18.166.254:35987)
0812 05:16:13.044517 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:13.044519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:13.044532 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:16:13.044589 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:13.044590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:13.044603 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:13.044672 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:13.045004 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.045822 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.046775 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.046974 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.047039 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:13.049874 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:39237: BlockingRecv error: recv got EOF from 127.18.166.195:39237 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 05:16:13 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:13 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:13.764775 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:13.759697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:50125)
0812 05:16:13.759850 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:13.759865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:13.759956 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:13.760117 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:13.760118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:13.760300 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:13.760351 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:13.760353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.761106 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.761107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:13.761716 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:13.761718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.762106 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.762107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:13.762153 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:13.762736 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:13.762741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:13.764711 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
W20260812 05:16:13.764981 28744 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:13.759815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:50125 (local address 127.18.166.254:35987)
0812 05:16:13.759947 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:13.759949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:13.759960 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:13.760018 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:13.760019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:13.760029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:13.760085 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:13.760461 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.760990 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.761814 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.762027 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.762097 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:13.764902 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:50125: BlockingRecv error: recv got EOF from 127.18.166.193:50125 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
Aug 12 05:16:13 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:13 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:13.916018 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:13.910762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:58771)
0812 05:16:13.910972 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:13.910988 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:13.911096 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:13.911286 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:13.911287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:13.911496 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:13.911548 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:13.911551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.912247 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.912249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:13.912809 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:13.912811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.913216 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.913218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:13.913286 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:13.913903 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:13.913908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:13.915954 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":101,"threads_started":1}
W20260812 05:16:13.916215 28744 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:13.910892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:58771 (local address 127.18.166.254:35987)
0812 05:16:13.910988 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:13.910991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:13.911074 (+ 83us) server_negotiation.cc:407] Connection header received
0812 05:16:13.911183 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:13.911184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:13.911197 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:13.911285 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:13.911644 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.912150 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.912913 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:13.913120 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:13.913203 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:13.916144 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:58771: BlockingRecv error: recv got EOF from 127.18.166.194:58771 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 05:16:14 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:14 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:14.055785 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:14.050480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:35155)
0812 05:16:14.050665 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:14.050678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:14.050742 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:14.050913 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:14.050914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:14.051093 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:14.051144 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:14.051145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.051963 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.051964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:14.052539 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:14.052541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.052930 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.052931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:14.052978 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:14.053635 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:14.053641 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:14.055719 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":130,"threads_started":1}
W20260812 05:16:14.055977 28744 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:14.050671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:35155 (local address 127.18.166.254:35987)
0812 05:16:14.050747 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:14.050750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:14.050764 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:16:14.050807 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:14.050807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:14.050821 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:14.050892 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:14.051232 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.051775 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.052622 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.052845 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.052915 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:14.055867 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:35155: BlockingRecv error: recv got EOF from 127.18.166.195:35155 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 05:16:14 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:14 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:14.773999 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:14.765309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:38513)
0812 05:16:14.765578 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:14.765588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:14.765690 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:14.765872 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:14.765873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:14.766080 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:14.766134 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:14.766136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.766851 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.766853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:14.767397 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:14.767399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.767828 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.767830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:14.767894 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:14.771650 (+ 3756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:14.771658 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:14.773934 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
W20260812 05:16:14.774195 28748 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:14.765364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:38513 (local address 127.18.166.254:35987)
0812 05:16:14.765524 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:14.765526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:14.765680 (+ 154us) server_negotiation.cc:407] Connection header received
0812 05:16:14.765762 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:14.765763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:14.765777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:14.765863 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:14.766234 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.766744 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.767497 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.767710 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.767826 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:14.774075 (+ 6249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:38513: BlockingRecv error: recv got EOF from 127.18.166.193:38513 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
Aug 12 05:16:14 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:14 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:14.922300 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:14.916670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:37587)
0812 05:16:14.916944 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:14.916960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:14.917093 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:14.917265 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:14.917265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:14.917505 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:14.917561 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:14.917564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.918305 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.918307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:14.918875 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:14.918877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.919401 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.919402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:14.919490 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:14.920118 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:14.920124 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:14.922213 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":132,"threads_started":1}
W20260812 05:16:14.922401 28748 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:14.916818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:37587 (local address 127.18.166.254:35987)
0812 05:16:14.916951 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:14.916954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:14.917068 (+ 114us) server_negotiation.cc:407] Connection header received
0812 05:16:14.917182 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:14.917183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:14.917196 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:14.917250 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:14.917662 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.918186 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.918974 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:14.919306 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:14.919399 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:14.922335 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:37587: BlockingRecv error: recv got EOF from 127.18.166.194:37587 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 05:16:15 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:15 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:15.061993 28750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:15.056558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:49955)
0812 05:16:15.056770 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:15.056782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:15.056855 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:15.056993 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:15.056994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:15.057167 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:15.057223 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:15.057225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.058028 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.058029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:15.058576 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:15.058578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.059017 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.059018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:15.059087 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:15.059651 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:15.059657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:15.061867 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":117,"threads_started":1}
W20260812 05:16:15.062075 28748 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:15.056646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:49955 (local address 127.18.166.254:35987)
0812 05:16:15.056726 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:15.056728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:15.056858 (+ 130us) server_negotiation.cc:407] Connection header received
0812 05:16:15.056902 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:15.056903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:15.056916 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:15.056970 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:15.057338 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.057889 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.058712 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.058907 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.058977 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:15.062011 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:49955: BlockingRecv error: recv got EOF from 127.18.166.195:49955 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 05:16:15 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:15 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:15.779526 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:15.774626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:35887)
0812 05:16:15.774797 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:15.774809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:15.774878 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:15.775042 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:15.775043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:15.775232 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:15.775284 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:15.775286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.775942 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.775942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:15.776499 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:15.776500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.776878 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.776879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:15.776922 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:15.777451 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:15.777456 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:15.779461 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
W20260812 05:16:15.779732 28752 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:15.774751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:35887 (local address 127.18.166.254:35987)
0812 05:16:15.774896 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:15.774898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:15.774909 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:15.774942 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:15.774943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:15.774954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:15.775014 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:15.775362 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.775864 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.776591 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.776801 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.776861 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:15.779613 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:35887: BlockingRecv error: recv got EOF from 127.18.166.193:35887 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
Aug 12 05:16:15 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:15 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:15.928546 28753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:15.922999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:49467)
0812 05:16:15.923270 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:15.923284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:15.923359 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:15.923576 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:15.923577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:15.923765 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:15.923819 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:15.923821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.924531 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.924532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:15.925096 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:15.925099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.925531 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.925532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:15.925588 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:15.926198 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:15.926203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:15.928480 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":160,"threads_started":1}
W20260812 05:16:15.928754 28752 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:15.923216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:49467 (local address 127.18.166.254:35987)
0812 05:16:15.923299 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:15.923301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:15.923348 (+ 47us) server_negotiation.cc:407] Connection header received
0812 05:16:15.923430 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:15.923431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:15.923445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:15.923516 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:15.923900 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.924417 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.925187 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:15.925382 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:15.925519 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:15.928677 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:49467: BlockingRecv error: recv got EOF from 127.18.166.194:49467 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 05:16:16 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:16 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:16.067565 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:16.062580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:54139)
0812 05:16:16.062798 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:16.062824 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:16.062901 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:16.063093 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:16.063094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:16.063306 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:16.063358 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:16.063359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.064071 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.064072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:16.064611 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:16.064612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.064991 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.064992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:16.065052 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:16.065581 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:16.065585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:16.067462 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":118,"threads_started":1}
W20260812 05:16:16.067626 28752 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:16.062661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:54139 (local address 127.18.166.254:35987)
0812 05:16:16.062740 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:16.062742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:16.062883 (+ 141us) server_negotiation.cc:407] Connection header received
0812 05:16:16.062987 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:16.062988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:16.063003 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:16.063084 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:16.063447 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.063964 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.064705 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.064911 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.064992 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:16.067560 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:54139: BlockingRecv error: recv got EOF from 127.18.166.195:54139 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 05:16:16 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:16 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:16.785219 28755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:16.780185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:53657)
0812 05:16:16.780338 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:16.780352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:16.780425 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:16.780640 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:16.780641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:16.780835 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:16.780887 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:16.780888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.781614 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.781616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:16.782166 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:16.782168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.782544 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.782545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:16.782590 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:16.783117 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:16.783122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:16.785154 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
W20260812 05:16:16.785375 28756 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:16.780283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:53657 (local address 127.18.166.254:35987)
0812 05:16:16.780468 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:16.780470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:16.780478 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:16.780526 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:16.780526 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:16.780540 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:16.780611 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:16.780971 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.781524 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.782267 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.782469 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.782525 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:16.785279 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:53657: BlockingRecv error: recv got EOF from 127.18.166.193:53657 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
Aug 12 05:16:16 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:16 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:16.934752 28757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:16.929267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:46545)
0812 05:16:16.929481 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:16.929493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:16.929598 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:16.929790 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:16.929791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:16.930042 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:16.930156 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:16.930159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.930905 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.930907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:16.931456 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:16.931458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.931856 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.931857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:16.931920 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:16.932486 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:16.932492 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:16.934636 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
W20260812 05:16:16.934857 28756 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:16.929344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:46545 (local address 127.18.166.254:35987)
0812 05:16:16.929434 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:16.929437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:16.929575 (+ 138us) server_negotiation.cc:407] Connection header received
0812 05:16:16.929684 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:16.929685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:16.929705 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:16.929767 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:16.930244 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.930814 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.931559 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:16.931757 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:16.931840 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:16.934786 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:46545: BlockingRecv error: recv got EOF from 127.18.166.194:46545 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 05:16:17 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:17 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:17.073014 28758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:17.068117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:35013)
0812 05:16:17.068296 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:17.068316 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:17.068369 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:17.068496 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:17.068497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:17.068683 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:17.068736 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:17.068737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.069370 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.069370 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:17.069949 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:17.069951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.070301 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.070302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:17.070346 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:17.070896 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:17.070901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:17.072907 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
W20260812 05:16:17.073068 28756 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:17.068239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:35013 (local address 127.18.166.254:35987)
0812 05:16:17.068342 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:17.068343 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:17.068370 (+ 27us) server_negotiation.cc:407] Connection header received
0812 05:16:17.068421 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:17.068421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:17.068434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:17.068475 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:17.068804 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.069299 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.070029 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.070232 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.070288 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:17.072999 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:35013: BlockingRecv error: recv got EOF from 127.18.166.195:35013 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 05:16:17 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:17 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:17.791327 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:17.786081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:35581)
0812 05:16:17.786237 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:17.786250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:17.786372 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:17.786525 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:17.786526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:17.786705 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:17.786760 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:17.786762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.787565 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.787565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:17.788135 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:17.788136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.788548 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.788549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:17.788601 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:17.789149 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:17.789154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:17.791233 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
W20260812 05:16:17.791471 28760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:17.786149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:35581 (local address 127.18.166.254:35987)
0812 05:16:17.786266 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:17.786268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:17.786359 (+ 91us) server_negotiation.cc:407] Connection header received
0812 05:16:17.786435 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:17.786435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:17.786447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:17.786507 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:17.786867 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.787472 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.788237 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.788461 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.788529 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:17.791353 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:35581: BlockingRecv error: recv got EOF from 127.18.166.193:35581 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
Aug 12 05:16:17 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:17 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:17.940698 28761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:17.935471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:44937)
0812 05:16:17.935666 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:17.935678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:17.935759 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:17.935900 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:17.935901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:17.936086 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:17.936141 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:17.936142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.936836 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.936837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:17.937366 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:17.937368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.937786 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.937786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:17.937835 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:17.938339 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:17.938345 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:17.940579 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":109,"threads_started":1}
W20260812 05:16:17.940758 28760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:17.935545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:44937 (local address 127.18.166.254:35987)
0812 05:16:17.935625 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:17.935627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:17.935756 (+ 129us) server_negotiation.cc:407] Connection header received
0812 05:16:17.935821 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:17.935821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:17.935834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:17.935878 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:17.936242 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.936756 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.937514 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:17.937714 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:17.937775 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:17.940687 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:44937: BlockingRecv error: recv got EOF from 127.18.166.194:44937 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 05:16:18 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:18 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:18.078962 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:18.073744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:58267)
0812 05:16:18.073988 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:18.074001 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:18.074129 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:18.074309 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:18.074311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:18.074588 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:18.074659 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:18.074662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.075347 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.075348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:18.075891 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:18.075893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.076239 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.076240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:18.076289 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:18.076847 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:18.076853 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:18.078899 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":118,"threads_started":1}
W20260812 05:16:18.079108 28760 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:18.073877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:58267 (local address 127.18.166.254:35987)
0812 05:16:18.073947 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:18.073949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:18.074122 (+ 173us) server_negotiation.cc:407] Connection header received
0812 05:16:18.074193 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:18.074193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:18.074210 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:18.074289 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:18.074740 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.075255 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.075980 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.076168 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.076222 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:18.079019 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:58267: BlockingRecv error: recv got EOF from 127.18.166.195:58267 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 05:16:18 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:18 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:18.797171 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:18.791854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:48193)
0812 05:16:18.791995 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:18.792013 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:18.792067 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:18.792277 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:18.792277 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:18.792478 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:18.792534 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:18.792535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.793271 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.793272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:18.793923 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:18.793925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.794335 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.794336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:18.794392 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:18.794962 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:18.794968 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:18.797077 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
W20260812 05:16:18.797276 28764 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:18.792001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:48193 (local address 127.18.166.254:35987)
0812 05:16:18.792135 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:18.792137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:18.792145 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:16:18.792169 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:18.792170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:18.792179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:18.792248 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:18.792637 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.793165 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.794002 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.794222 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.794288 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:18.797198 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:48193: BlockingRecv error: recv got EOF from 127.18.166.193:48193 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
Aug 12 05:16:18 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:18 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:18.946542 28765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:18.941237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:42519)
0812 05:16:18.941381 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:18.941392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:18.941472 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:18.941674 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:18.941674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:18.941851 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:18.941904 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:18.941906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.942743 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.942744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:18.943286 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:18.943287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.943718 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.943719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:18.943770 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:18.944289 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:18.944295 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:18.946438 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
W20260812 05:16:18.946607 28764 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:18.941458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:42519 (local address 127.18.166.254:35987)
0812 05:16:18.941526 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:18.941528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:18.941539 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:18.941572 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:18.941573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:18.941585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:18.941657 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:18.941986 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.942648 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.943394 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:18.943641 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:18.943704 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:18.946540 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:42519: BlockingRecv error: recv got EOF from 127.18.166.194:42519 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 05:16:19 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:19 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:19.084592 28766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:19.079527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:43317)
0812 05:16:19.079675 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:19.079692 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:19.079755 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:19.079884 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:19.079885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:19.080062 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:19.080113 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:19.080115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.080860 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.080861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:19.081390 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:19.081391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.081819 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.081820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:19.081868 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:19.082361 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:19.082365 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:19.084469 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
W20260812 05:16:19.084671 28764 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:19.079588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:43317 (local address 127.18.166.254:35987)
0812 05:16:19.079649 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:19.079651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:19.079742 (+ 91us) server_negotiation.cc:407] Connection header received
0812 05:16:19.079799 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:19.079800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:19.079815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:19.079864 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:19.080218 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.080776 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.081529 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.081748 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.081809 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:19.084598 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:43317: BlockingRecv error: recv got EOF from 127.18.166.195:43317 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 05:16:19 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:19 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:19.806288 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:19.797759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:50745)
0812 05:16:19.797929 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:19.797942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:19.798012 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:19.798153 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:19.798154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:19.798347 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:19.798407 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:19.798409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.799047 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.799048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:19.799583 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:19.799584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.799969 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.799970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:19.800067 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:19.804002 (+ 3935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:19.804009 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:19.806182 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
W20260812 05:16:19.806406 28768 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:19.797842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:50745 (local address 127.18.166.254:35987)
0812 05:16:19.797956 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:19.797958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:19.797992 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:16:19.798067 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:19.798067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:19.798078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:19.798138 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:19.798472 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.798976 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.799664 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.799869 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.799947 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:19.806329 (+ 6382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:50745: BlockingRecv error: recv got EOF from 127.18.166.193:50745 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
Aug 12 05:16:19 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:19 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:19.953792 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:19.947165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:56687)
0812 05:16:19.947334 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:19.947347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:19.947469 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:19.947652 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:19.947654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:19.947929 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:19.947989 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:19.947991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.949032 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.949034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:19.949879 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:19.949884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.950418 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.950419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:19.950482 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:19.951064 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:19.951070 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:19.953693 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
W20260812 05:16:19.953894 28768 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:19.947291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:56687 (local address 127.18.166.254:35987)
0812 05:16:19.947359 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:19.947362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:19.947441 (+ 79us) server_negotiation.cc:407] Connection header received
0812 05:16:19.947535 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:19.947536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:19.947552 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:19.947622 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:19.948109 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.948920 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.949967 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:19.950261 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:19.950342 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:19.953820 (+ 3478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:56687: BlockingRecv error: recv got EOF from 127.18.166.194:56687 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 05:16:20 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:20 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:20.090327 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:20.085153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:41613)
0812 05:16:20.085330 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:20.085342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:20.085457 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:20.085692 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:20.085693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:20.085887 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:20.085940 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:20.085942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.086737 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.086738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:20.087292 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:20.087295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.087682 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.087682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:20.087733 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:20.088229 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:20.088234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:20.090263 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
W20260812 05:16:20.090493 28768 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:20.085291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:41613 (local address 127.18.166.254:35987)
0812 05:16:20.085398 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:20.085400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:20.085433 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:16:20.085555 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:20.085556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:20.085573 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:20.085657 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:20.086041 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.086618 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.087394 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.087594 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.087668 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:20.090399 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:41613: BlockingRecv error: recv got EOF from 127.18.166.195:41613 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 05:16:20 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:20 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:20.812340 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:20.807021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:42575)
0812 05:16:20.807192 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:20.807203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:20.807259 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:20.807476 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:20.807476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:20.807658 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:20.807713 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:20.807715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.808389 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.808390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:20.809140 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:20.809144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.809691 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.809692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:20.809747 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:20.810303 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:20.810309 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:20.812276 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
W20260812 05:16:20.812542 28772 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:20.807139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:42575 (local address 127.18.166.254:35987)
0812 05:16:20.807314 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:20.807316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:20.807326 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:20.807356 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:20.807356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:20.807368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:20.807460 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:20.807798 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.808306 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.809277 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.809617 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.809679 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:20.812453 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:42575: BlockingRecv error: recv got EOF from 127.18.166.193:42575 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
Aug 12 05:16:20 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:20 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:20.960558 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:20.954488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:58791)
0812 05:16:20.954686 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:20.954700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:20.954794 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:20.954983 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:20.954984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:20.955228 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:20.955308 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:20.955311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.956321 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.956322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:20.956854 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:20.956856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.957447 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.957449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:20.957518 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:20.958121 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:20.958127 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:20.960467 (+ 2340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
W20260812 05:16:20.960726 28772 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:20.954546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:58791 (local address 127.18.166.254:35987)
0812 05:16:20.954616 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:20.954618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:20.954762 (+ 144us) server_negotiation.cc:407] Connection header received
0812 05:16:20.954876 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:20.954878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:20.954892 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:20.954959 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:20.955398 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.956229 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.956941 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:20.957313 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:20.957385 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:20.960627 (+ 3242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:58791: BlockingRecv error: recv got EOF from 127.18.166.194:58791 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 05:16:21 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:21 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:21.096261 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:21.091035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:39651)
0812 05:16:21.091182 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:21.091195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:21.091264 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:21.091466 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:21.091467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:21.091649 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:21.091704 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:21.091706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.092377 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.092378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:21.092914 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:21.092916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.093250 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.093250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:21.093292 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:21.093788 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:21.093794 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:21.096134 (+ 2340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
W20260812 05:16:21.096328 28772 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:21.091200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39651 (local address 127.18.166.254:35987)
0812 05:16:21.091318 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:21.091321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:21.091332 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:21.091362 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:21.091362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:21.091375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:21.091445 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:21.091787 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.092297 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.092998 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.093183 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.093253 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:21.096260 (+ 3007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:39651: BlockingRecv error: recv got EOF from 127.18.166.195:39651 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 05:16:21 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:21 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:21.817756 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:21.812864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:36303)
0812 05:16:21.813036 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:21.813048 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:21.813106 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:21.813264 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:21.813264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:21.813493 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:21.813543 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:21.813545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.814234 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.814235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:21.814804 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:21.814806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.815192 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.815192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:21.815234 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:21.815752 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:21.815757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:21.817685 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
W20260812 05:16:21.817934 28776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:21.812962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:36303 (local address 127.18.166.254:35987)
0812 05:16:21.813077 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:21.813079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:21.813091 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:16:21.813166 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:21.813166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:21.813177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:21.813235 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:21.813623 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.814130 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.814905 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.815110 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.815181 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:21.817809 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:36303: BlockingRecv error: recv got EOF from 127.18.166.193:36303 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
Aug 12 05:16:21 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:21 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:21.966244 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:21.961191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:37111)
0812 05:16:21.961343 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:21.961360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:21.961487 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:21.961645 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:21.961646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:21.961840 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:21.961893 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:21.961894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.962552 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.962553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:21.963111 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:21.963113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.963457 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.963458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:21.963506 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:21.964101 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:21.964107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:21.966174 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
W20260812 05:16:21.966372 28776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:21.961266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:37111 (local address 127.18.166.254:35987)
0812 05:16:21.961347 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:21.961349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:21.961450 (+ 101us) server_negotiation.cc:407] Connection header received
0812 05:16:21.961549 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:21.961550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:21.961561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:21.961629 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:21.961968 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.962468 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.963188 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:21.963384 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:21.963440 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:21.966291 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:37111: BlockingRecv error: recv got EOF from 127.18.166.194:37111 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 05:16:22 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:22 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:22.102031 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:22.096906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:53071)
0812 05:16:22.097075 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:22.097088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:22.097161 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:22.097344 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:22.097346 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:22.097621 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:22.097686 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:22.097688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.098387 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.098387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:22.098939 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:22.098941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.099308 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.099309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:22.099358 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:22.099941 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:22.099946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:22.101944 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
W20260812 05:16:22.102134 28776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:22.097047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:53071 (local address 127.18.166.254:35987)
0812 05:16:22.097130 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:22.097133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:22.097144 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:22.097225 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:22.097226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:22.097247 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:22.097320 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:22.097764 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.098305 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.099018 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.099236 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.099291 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:22.102062 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:53071: BlockingRecv error: recv got EOF from 127.18.166.195:53071 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 05:16:22 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:22 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:22.823665 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:22.818443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:48007)
0812 05:16:22.818663 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:22.818676 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:22.818756 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:22.818928 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:22.818929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:22.819138 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:22.819188 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:22.819189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.819841 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.819842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:22.820403 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:22.820404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.820762 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.820763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:22.820813 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:22.821388 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:22.821393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:22.823544 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":124,"threads_started":1}
W20260812 05:16:22.823755 28780 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:22.818541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:48007 (local address 127.18.166.254:35987)
0812 05:16:22.818666 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:22.818668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:22.818760 (+ 92us) server_negotiation.cc:407] Connection header received
0812 05:16:22.818816 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:22.818817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:22.818831 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:22.818912 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:22.819266 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.819770 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.820485 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.820687 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.820741 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:22.823671 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:48007: BlockingRecv error: recv got EOF from 127.18.166.193:48007 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
Aug 12 05:16:22 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:22 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:22.971951 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:22.966829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:58897)
0812 05:16:22.967009 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:22.967020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:22.967119 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:22.967259 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:22.967260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:22.967449 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:22.967502 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:22.967504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.968177 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.968178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:22.968723 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:22.968724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.969113 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.969114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:22.969169 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:22.969873 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:22.969878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:22.971883 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
W20260812 05:16:22.972159 28780 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:22.966875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:58897 (local address 127.18.166.254:35987)
0812 05:16:22.966950 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:22.966952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:22.967122 (+ 170us) server_negotiation.cc:407] Connection header received
0812 05:16:22.967174 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:22.967175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:22.967186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:22.967239 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:22.967582 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.968087 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.968814 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:22.969014 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:22.969073 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:22.972091 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:58897: BlockingRecv error: recv got EOF from 127.18.166.194:58897 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 05:16:23 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:23 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:23.107559 28782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:23.102614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:58491)
0812 05:16:23.102830 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:23.102846 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:23.102923 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:23.103073 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:23.103074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:23.103259 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:23.103316 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:23.103318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.104006 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.104007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:23.104593 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:23.104595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.104973 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.104974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:23.105020 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:23.105582 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:23.105587 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:23.107487 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
W20260812 05:16:23.107733 28780 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:23.102815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:58491 (local address 127.18.166.254:35987)
0812 05:16:23.102888 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:23.102890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:23.102904 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:16:23.102988 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:23.102989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:23.103002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:23.103060 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:23.103387 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.103906 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.104696 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.104895 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.104959 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:23.107647 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:58491: BlockingRecv error: recv got EOF from 127.18.166.195:58491 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 05:16:23 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:23 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:23.829460 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:23.824321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:43463)
0812 05:16:23.824503 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:23.824515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:23.824580 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:23.824730 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:23.824730 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:23.824920 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:23.824973 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:23.824974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.825739 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.825740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:23.826284 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:23.826285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.826662 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.826663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:23.826704 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:23.827208 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:23.827212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:23.829291 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
W20260812 05:16:23.829612 28784 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:23.824464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:43463 (local address 127.18.166.254:35987)
0812 05:16:23.824582 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:23.824584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:23.824617 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:16:23.824644 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:23.824645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:23.824656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:23.824709 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:23.825052 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.825648 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.826369 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.826586 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.826650 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:23.829491 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:43463: BlockingRecv error: recv got EOF from 127.18.166.193:43463 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
Aug 12 05:16:23 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:23 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:23.977377 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:23.972463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:40399)
0812 05:16:23.972602 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:23.972613 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:23.972662 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:23.972798 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:23.972798 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:23.972979 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:23.973032 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:23.973033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.973732 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.973733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:23.974276 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:23.974278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.974629 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.974630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:23.974692 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:23.975309 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:23.975314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:23.977276 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
W20260812 05:16:23.977490 28784 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:23.972616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:40399 (local address 127.18.166.254:35987)
0812 05:16:23.972679 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:23.972681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:23.972691 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:23.972721 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:23.972722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:23.972731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:23.972786 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:23.973096 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.973650 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.974346 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:23.974533 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:23.974586 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:23.977384 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:40399: BlockingRecv error: recv got EOF from 127.18.166.194:40399 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 05:16:24 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:24 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:24.112797 28786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:24.108083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:46767)
0812 05:16:24.108259 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:24.108273 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:24.108345 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:24.108492 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:24.108493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:24.108682 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:24.108742 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:24.108743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.109398 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.109400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:24.110003 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:24.110006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.110341 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.110342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:24.110383 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:24.110922 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:24.110927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:24.112737 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
W20260812 05:16:24.112916 28784 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:24.108228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:46767 (local address 127.18.166.254:35987)
0812 05:16:24.108295 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:24.108297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:24.108326 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:16:24.108402 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:24.108403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:24.108415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:24.108468 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:24.108809 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.109317 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.110083 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.110273 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.110328 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:24.112839 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:46767: BlockingRecv error: recv got EOF from 127.18.166.195:46767 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 05:16:24 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:24 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:24.840337 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:24.830013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:46537)
0812 05:16:24.830193 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:24.830209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:24.830267 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:24.830494 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:24.830495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:24.830687 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:24.830740 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:24.830742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.831480 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.831481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:24.832082 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:24.832085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.832505 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.832506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:24.832596 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:24.838193 (+ 5597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:24.838201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:24.840261 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
W20260812 05:16:24.840547 28788 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:24.830125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:46537 (local address 127.18.166.254:35987)
0812 05:16:24.830282 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:24.830284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:24.830295 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:16:24.830325 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:24.830326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:24.830337 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:24.830432 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:24.830830 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.831359 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.832198 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.832423 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.832493 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:24.840474 (+ 7981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:46537: BlockingRecv error: recv got EOF from 127.18.166.193:46537 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
Aug 12 05:16:24 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:24 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:24.984215 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:24.978230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:49563)
0812 05:16:24.978467 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:24.978480 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:24.978563 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:24.978855 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:24.978857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:24.979199 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:24.979258 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:24.979261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.980009 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.980010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:24.980588 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:24.980591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.981010 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.981011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:24.981090 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:24.981763 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:24.981769 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:24.984123 (+ 2354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":146,"threads_started":1}
W20260812 05:16:24.984391 28788 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:24.978353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:49563 (local address 127.18.166.254:35987)
0812 05:16:24.978444 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:24.978447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:24.978563 (+ 116us) server_negotiation.cc:407] Connection header received
0812 05:16:24.978631 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:24.978632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:24.978644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:24.978758 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:24.979373 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.979896 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.980705 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:24.980910 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:24.981005 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:24.984291 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:49563: BlockingRecv error: recv got EOF from 127.18.166.194:49563 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 05:16:25 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:25 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.195@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:25.118341 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:25.113556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.195:44665)
0812 05:16:25.113687 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:25.113699 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:25.113755 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:25.113891 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:25.113891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:25.114082 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:25.114141 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:25.114143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.114817 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.114818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:25.115380 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:25.115382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.115706 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.115706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:25.115749 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:25.116210 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:25.116214 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:25.118237 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
W20260812 05:16:25.118425 28788 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:25.113634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:44665 (local address 127.18.166.254:35987)
0812 05:16:25.113722 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:25.113724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:25.113738 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:16:25.113815 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:25.113815 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:25.113829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:25.113872 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:25.114206 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.114728 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.115460 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.115641 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.115693 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:25.118346 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.195:44665: BlockingRecv error: recv got EOF from 127.18.166.195:44665 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 05:16:25 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:25 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.193@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:25.845857 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:25.840890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.193:44063)
0812 05:16:25.841052 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:25.841065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:25.841120 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:25.841310 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:25.841311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:25.841509 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:25.841560 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:25.841562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.842332 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.842333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:25.842856 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:25.842858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.843249 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.843250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:25.843293 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:25.843815 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:25.843820 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:25.845789 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
W20260812 05:16:25.846063 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:25.841011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:44063 (local address 127.18.166.254:35987)
0812 05:16:25.841152 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:25.841154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:25.841161 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:16:25.841184 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:25.841185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:25.841196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:25.841284 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:25.841656 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.842230 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.842943 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.843174 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.843233 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:25.845982 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.193:44063: BlockingRecv error: recv got EOF from 127.18.166.193:44063 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
Aug 12 05:16:25 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:16:25 dist-test-slave-srgv krb5kdc[27658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.166.194@KRBTEST.COM for oryx/127.18.166.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:16:25.990005 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:25.984820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:35987 (local address 127.18.166.194:54175)
0812 05:16:25.984965 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:25.984977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:25.985073 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:25.985240 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:25.985241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:25.985519 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:25.985573 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:25.985576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.986342 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.986343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:25.986917 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:25.986919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.987356 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.987358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:25.987419 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:25.988000 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:25.988005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:25.989941 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.166.254:35987: Server oryx/127.18.166.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":109,"threads_started":1}
W20260812 05:16:25.990193 28793 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:25.984892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:54175 (local address 127.18.166.254:35987)
0812 05:16:25.984979 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:25.984982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:25.985059 (+ 77us) server_negotiation.cc:407] Connection header received
0812 05:16:25.985144 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:25.985145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:25.985156 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:25.985208 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:25.985669 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.986256 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.986989 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:25.987239 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:25.987302 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:25.990110 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.166.194:54175: BlockingRecv error: recv got EOF from 127.18.166.194:54175 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 05:16:26.027916 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28292
I20260812 05:16:26.034561 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28457
I20260812 05:16:26.040396 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28595
I20260812 05:16:26.048050 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28218
2026-08-12T05:16:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17688 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T05:16:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:26Z Disabled control of system clock
I20260812 05:16:26.082859 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42067
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:42067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 05:16:26.173290 28798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:26.173578 28798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:26.173602 28798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:26.174932 28798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:26.174988 28798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:26.175001 28798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:26.175015 28798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:26.175027 28798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:26.176505 28798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:42067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:26.176779 28798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:26.177090 28798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:26.183113 28805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.183120 28804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.183286 28798 server_base.cc:1034] running on GCE node
W20260812 05:16:26.183120 28807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.183991 28798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:26.184571 28798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:26.185984 28798 hybrid_clock.cc:648] HybridClock initialized: now 1786511786185961 us; error 48 us; skew 500 ppm
I20260812 05:16:26.187714 28798 webserver.cc:460] Webserver started at http://127.18.166.254:37375/ using document root <none> and password file <none>
I20260812 05:16:26.188027 28798 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:26.188108 28798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:26.188287 28798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:26.189749 28798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "9314ea013ac948db92607971061270f5"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "644b2e1fc8ece80c67966e9882baab92"
server_key_iv: "fe745dcaa36b58b06f6fc8d1b845d4c8"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.190193 28798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "9314ea013ac948db92607971061270f5"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "644b2e1fc8ece80c67966e9882baab92"
server_key_iv: "fe745dcaa36b58b06f6fc8d1b845d4c8"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.191807 28798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:16:26.192800 28814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.193128 28798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:26.193287 28798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "9314ea013ac948db92607971061270f5"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "644b2e1fc8ece80c67966e9882baab92"
server_key_iv: "fe745dcaa36b58b06f6fc8d1b845d4c8"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.193388 28798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:26.214968 28798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:26.215489 28798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:26.215674 28798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:26.220324 28798 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:42067
I20260812 05:16:26.220377 28871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:42067 every 8 connection(s)
I20260812 05:16:26.220803 28798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:26.221756 28872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:26.224822 28872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5: Bootstrap starting.
I20260812 05:16:26.225580 28872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:26.225940 28872 log.cc:824] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:26.226836 28872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5: No bootstrap required, opened a new log
I20260812 05:16:26.228677 28872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9314ea013ac948db92607971061270f5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42067 } }
I20260812 05:16:26.228863 28872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:26.228886 28872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9314ea013ac948db92607971061270f5, State: Initialized, Role: FOLLOWER
I20260812 05:16:26.229067 28872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9314ea013ac948db92607971061270f5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42067 } }
I20260812 05:16:26.229151 28872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:26.229193 28872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:26.229259 28872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:26.229583 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28798
I20260812 05:16:26.229734 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:16:26.229934 19099 external_mini_cluster.cc:1428] Setting key 4e610435e2c6c2264dbc44b2a89081b8
I20260812 05:16:26.230156 28872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9314ea013ac948db92607971061270f5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42067 } }
I20260812 05:16:26.230302 28872 leader_election.cc:302] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9314ea013ac948db92607971061270f5; no voters:
I20260812 05:16:26.230507 28872 leader_election.cc:288] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:16:26.230620 28876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:26.230804 28876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [term 1 LEADER]: Becoming Leader. State: Replica: 9314ea013ac948db92607971061270f5, State: Running, Role: LEADER
I20260812 05:16:26.230923 28872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:26.230945 28876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9314ea013ac948db92607971061270f5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42067 } }
I20260812 05:16:26.231498 28877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9314ea013ac948db92607971061270f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9314ea013ac948db92607971061270f5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42067 } } }
I20260812 05:16:26.231616 28877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:26.231645 28881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9314ea013ac948db92607971061270f5. Latest consensus state: current_term: 1 leader_uuid: "9314ea013ac948db92607971061270f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9314ea013ac948db92607971061270f5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42067 } } }
I20260812 05:16:26.231735 28881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:26.231988 28890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:26.232697 28890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:16:26.234839 28890 catalog_manager.cc:1347] Generated new cluster ID: 2d88bef409bf43088d778cb6c984e9c7
I20260812 05:16:26.234916 28890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:26.237991 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.230558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.18.166.254:42067)
0812 05:16:26.230998 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:26.231009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:26.231044 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:16:26.231181 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:26.231195 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:26.231264 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:26.232893 (+ 1629us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:26.233290 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.234685 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.235650 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.236125 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.236267 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.237161 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:26.237165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:26.237166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:26.237371 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:26.237456 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:26.237472 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:26.237546 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:26.237664 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:16:26.237768 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":184,"threads_started":1}
I20260812 05:16:26.269742 28890 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:16:26.270362 28890 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:26.272385 28890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9314ea013ac948db92607971061270f5: Generated new TSK 0
I20260812 05:16:26.272586 28890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:16:26.282505 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42067
--builtin_ntp_servers=127.18.166.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 05:16:26.370520 28900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:26.370767 28900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:26.370790 28900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:26.372187 28900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:26.372247 28900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:26.372277 28900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:26.374905 28900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:26.375265 28900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:26.375636 28900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:26.381042 28909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.381037 28906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.381042 28907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.381331 28900 server_base.cc:1034] running on GCE node
I20260812 05:16:26.381649 28900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:26.381932 28900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:26.383097 28900 hybrid_clock.cc:648] HybridClock initialized: now 1786511786383085 us; error 30 us; skew 500 ppm
I20260812 05:16:26.384425 28900 webserver.cc:460] Webserver started at http://127.18.166.193:40037/ using document root <none> and password file <none>
I20260812 05:16:26.384675 28900 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:26.384723 28900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:26.384838 28900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:26.386040 28900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "493dd12a0a1a447ea620c673f9c8a158"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "dc23f1e54b31a5a7ba9ec22656e92901"
server_key_iv: "b63321b6989f1cf0de80ec38df939c0a"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.386373 28900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "493dd12a0a1a447ea620c673f9c8a158"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "dc23f1e54b31a5a7ba9ec22656e92901"
server_key_iv: "b63321b6989f1cf0de80ec38df939c0a"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.387764 28900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:16:26.388612 28915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.388803 28900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:26.388883 28900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "493dd12a0a1a447ea620c673f9c8a158"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "dc23f1e54b31a5a7ba9ec22656e92901"
server_key_iv: "b63321b6989f1cf0de80ec38df939c0a"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.388948 28900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:26.408766 28900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:26.409082 28900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:26.409195 28900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:26.409469 28900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:26.409838 28900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:26.409878 28900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.409902 28900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:26.409915 28900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.416641 28900 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:45885
I20260812 05:16:26.416675 29035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:45885 every 8 connection(s)
I20260812 05:16:26.417083 28900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:26.418998 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 28900
I20260812 05:16:26.419121 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:16:26.419332 19099 external_mini_cluster.cc:1428] Setting key f609dbcf611b8f8d90b4e80c7cc3032b
I20260812 05:16:26.420348 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42067
--builtin_ntp_servers=127.18.166.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 05:16:26.423261 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.417923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:54093 (local address 127.18.166.254:42067)
0812 05:16:26.418008 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:26.418010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:26.418272 (+ 262us) server_negotiation.cc:407] Connection header received
0812 05:16:26.418810 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:26.418812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:26.418829 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:26.418904 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:26.419889 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.420505 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.421634 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.421842 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.421931 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.422903 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:26.422905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:26.422906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:26.422915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:26.422937 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:26.422942 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:26.423003 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:26.423106 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 05:16:26.423147 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 05:16:26.423504 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.417909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42067 (local address 127.18.166.193:54093)
0812 05:16:26.418131 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:26.418171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:26.418701 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:26.418979 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:26.418982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:26.418991 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:26.419794 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:26.419800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.420632 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.420633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:26.421536 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:26.421539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.421947 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.421948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:26.422097 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.422741 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:16:26.422741 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:26.422772 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:16:26.422797 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:26.423003 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:26.423011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:26.423014 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:26.423107 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 05:16:26.423254 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260812 05:16:26.424072 29036 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42067
I20260812 05:16:26.424165 29036 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:26.424379 29036 heartbeater.cc:511] Master 127.18.166.254:42067 requested a full tablet report, sending...
I20260812 05:16:26.425001 28833 ts_manager.cc:194] Registered new tserver with Master: 493dd12a0a1a447ea620c673f9c8a158 (127.18.166.193:45885)
I20260812 05:16:26.426077 28833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.193:54093
W20260812 05:16:26.509057 29039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:26.509311 29039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:26.509335 29039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:26.510677 29039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:26.510739 29039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:26.510769 29039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:26.512789 29039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:26.513083 29039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:26.513399 29039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:26.518918 29044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.518918 29045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.519158 29047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.519052 29039 server_base.cc:1034] running on GCE node
I20260812 05:16:26.519423 29039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:26.519739 29039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:26.520915 29039 hybrid_clock.cc:648] HybridClock initialized: now 1786511786520905 us; error 37 us; skew 500 ppm
I20260812 05:16:26.522420 29039 webserver.cc:460] Webserver started at http://127.18.166.194:43197/ using document root <none> and password file <none>
I20260812 05:16:26.522715 29039 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:26.522789 29039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:26.522938 29039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:26.524124 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "a0ac6f03bb8348a3812d2cdd06dddfd9"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "2b6b2199aebcf7bcb6b4040eb6cb29de"
server_key_iv: "cb97b311af78613f391a181c6a76cc36"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.524510 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "a0ac6f03bb8348a3812d2cdd06dddfd9"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "2b6b2199aebcf7bcb6b4040eb6cb29de"
server_key_iv: "cb97b311af78613f391a181c6a76cc36"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.526140 29039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:16:26.527053 29053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.527233 29039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:26.527343 29039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "a0ac6f03bb8348a3812d2cdd06dddfd9"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "2b6b2199aebcf7bcb6b4040eb6cb29de"
server_key_iv: "cb97b311af78613f391a181c6a76cc36"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.527427 29039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:26.540551 29039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:26.540930 29039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:26.541085 29039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:26.541334 29039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:26.541710 29039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:26.541747 29039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.541790 29039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:26.541818 29039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.547991 29039 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:39091
I20260812 05:16:26.548084 29169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:39091 every 8 connection(s)
I20260812 05:16:26.548442 29039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:26.554076 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.549273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:36433 (local address 127.18.166.254:42067)
0812 05:16:26.549344 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:26.549346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:26.549616 (+ 270us) server_negotiation.cc:407] Connection header received
0812 05:16:26.550158 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:26.550159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:26.550171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:26.550264 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:26.551001 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.551507 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.552534 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.552734 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.552793 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.553806 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:26.553807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:26.553808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:26.553813 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:26.553829 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:26.553834 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:26.553881 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:26.553968 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 05:16:26.553998 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:16:26.554232 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.549261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42067 (local address 127.18.166.194:36433)
0812 05:16:26.549482 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:26.549526 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:26.550028 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:26.550366 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:26.550370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:26.550378 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:26.550916 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:26.550920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.551598 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.551599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:26.552427 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:26.552430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.552819 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.552820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:26.552942 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.553634 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:16:26.553635 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:26.553667 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:16:26.553693 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:26.553892 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:26.553897 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:26.553899 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:16:26.553955 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:16:26.554051 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260812 05:16:26.554873 29170 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42067
I20260812 05:16:26.555002 29170 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:26.555248 29170 heartbeater.cc:511] Master 127.18.166.254:42067 requested a full tablet report, sending...
I20260812 05:16:26.555641 28833 ts_manager.cc:194] Registered new tserver with Master: a0ac6f03bb8348a3812d2cdd06dddfd9 (127.18.166.194:39091)
I20260812 05:16:26.556043 28833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.194:36433
I20260812 05:16:26.556704 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29039
I20260812 05:16:26.556804 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:16:26.557029 19099 external_mini_cluster.cc:1428] Setting key 01410bb38496dd969c9e2e249ce103f4
I20260812 05:16:26.558019 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42067
--builtin_ntp_servers=127.18.166.212:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 05:16:26.648779 29173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:26.649044 29173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:26.649071 29173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:26.650367 29173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:26.650425 29173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:26.650454 29173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:26.651897 29173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:26.652140 29173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:26.652437 29173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:26.657924 29178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.658069 29173 server_base.cc:1034] running on GCE node
W20260812 05:16:26.657907 29181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.657912 29179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.658455 29173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:26.658795 29173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:26.659967 29173 hybrid_clock.cc:648] HybridClock initialized: now 1786511786659964 us; error 36 us; skew 500 ppm
I20260812 05:16:26.661401 29173 webserver.cc:460] Webserver started at http://127.18.166.195:40573/ using document root <none> and password file <none>
I20260812 05:16:26.661826 29173 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:26.661913 29173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:26.662070 29173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:26.663322 29173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "e7797b62a0e44720a5f4aa31087a4424"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "f76385e5f0c2722b60ab1a7d11ee191b"
server_key_iv: "4b33023c4bc74c499ecf07976ed4eefc"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.663766 29173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "e7797b62a0e44720a5f4aa31087a4424"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "f76385e5f0c2722b60ab1a7d11ee191b"
server_key_iv: "4b33023c4bc74c499ecf07976ed4eefc"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.665457 29173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 05:16:26.666435 29187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.666802 29173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:26.666950 29173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "e7797b62a0e44720a5f4aa31087a4424"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "f76385e5f0c2722b60ab1a7d11ee191b"
server_key_iv: "4b33023c4bc74c499ecf07976ed4eefc"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.667047 29173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:26.678715 29173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:26.679111 29173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:26.679282 29173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:26.679565 29173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:26.679941 29173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:26.679993 29173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.680035 29173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:26.680074 29173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.686550 29173 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:41859
I20260812 05:16:26.686600 29308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:41859 every 8 connection(s)
I20260812 05:16:26.686995 29173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:26.693035 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.687889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:45523 (local address 127.18.166.254:42067)
0812 05:16:26.687982 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:26.687984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:26.688204 (+ 220us) server_negotiation.cc:407] Connection header received
0812 05:16:26.688761 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:26.688762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:26.688775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:26.688849 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:26.689691 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.690208 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.691391 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.691591 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.691653 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.692698 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:26.692700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:26.692701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:26.692708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:26.692731 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:26.692736 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:26.692801 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:26.692902 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 05:16:26.692942 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 05:16:26.693293 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.687841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42067 (local address 127.18.166.195:45523)
0812 05:16:26.688064 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:26.688112 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:26.688640 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:26.688976 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:26.688992 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:26.689002 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:26.689579 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:26.689585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.690303 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.690304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:26.691289 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:26.691293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:26.691677 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:26.691678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:26.691803 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:26.692487 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:16:26.692487 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:26.692522 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:16:26.692567 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:26.692803 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:26.692811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:26.692815 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:26.692905 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:16:26.693027 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260812 05:16:26.693949 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29173
I20260812 05:16:26.693928 29309 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42067
I20260812 05:16:26.694033 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:16:26.694067 29309 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:26.694278 19099 external_mini_cluster.cc:1428] Setting key dd49afcfdae858014a8130573bc43331
I20260812 05:16:26.694286 29309 heartbeater.cc:511] Master 127.18.166.254:42067 requested a full tablet report, sending...
I20260812 05:16:26.694775 28832 ts_manager.cc:194] Registered new tserver with Master: e7797b62a0e44720a5f4aa31087a4424 (127.18.166.195:41859)
I20260812 05:16:26.695261 28832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.195:45523
I20260812 05:16:26.695360 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 05:16:26.697865 28879 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:26.697036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.18.166.254:42067)
0812 05:16:26.697107 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:26.697109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:26.697300 (+ 191us) server_negotiation.cc:407] Connection header received
0812 05:16:26.697394 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:26.697395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:26.697433 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:26.697488 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:26.697781 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55342: BlockingRecv error: recv got EOF from 127.0.0.1:55342 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:16:26.697872 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28900
I20260812 05:16:26.710328 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29039
I20260812 05:16:26.720023 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29173
I20260812 05:16:26.728108 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 28798
2026-08-12T05:16:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (676 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T05:16:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:26Z Disabled control of system clock
I20260812 05:16:26.759186 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:45607
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:45607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 05:16:26.845052 29320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:26.845293 29320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:26.845317 29320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:26.846637 29320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:26.846698 29320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:26.846712 29320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:26.846725 29320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:26.846737 29320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:26.848158 29320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:45607
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:45607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:26.848392 29320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:26.848681 29320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:26.853865 29328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.854013 29320 server_base.cc:1034] running on GCE node
W20260812 05:16:26.853895 29325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:26.853865 29326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:26.854384 29320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:26.854709 29320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:26.855911 29320 hybrid_clock.cc:648] HybridClock initialized: now 1786511786855882 us; error 48 us; skew 500 ppm
I20260812 05:16:26.857362 29320 webserver.cc:460] Webserver started at http://127.18.166.254:40905/ using document root <none> and password file <none>
I20260812 05:16:26.857682 29320 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:26.857735 29320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:26.857838 29320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:26.858932 29320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "c5064f778c5e49e29a10881a364e5446"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "172da17f36e308f79fd54fe2587b8ef2"
server_key_iv: "c0cb8e631217ce4a1d39893ef591ee9c"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.859251 29320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "c5064f778c5e49e29a10881a364e5446"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "172da17f36e308f79fd54fe2587b8ef2"
server_key_iv: "c0cb8e631217ce4a1d39893ef591ee9c"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.860656 29320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:16:26.861524 29334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:26.861709 29320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:26.861814 29320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "c5064f778c5e49e29a10881a364e5446"
format_stamp: "Formatted at 2026-08-12 05:16:26 on dist-test-slave-srgv"
server_key: "172da17f36e308f79fd54fe2587b8ef2"
server_key_iv: "c0cb8e631217ce4a1d39893ef591ee9c"
server_key_version: "encryptionkey@0"
I20260812 05:16:26.861899 29320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:26.865517 29320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:26.865841 29320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:26.865994 29320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:26.870347 29320 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:45607
I20260812 05:16:26.870386 29388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:45607 every 8 connection(s)
I20260812 05:16:26.870827 29320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:26.871570 29389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:26.874750 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29320
I20260812 05:16:26.874864 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:16:26.874892 29389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446: Bootstrap starting.
I20260812 05:16:26.875073 19099 external_mini_cluster.cc:1428] Setting key 3d078b551cc922ddb5ff65c87251a4d8
I20260812 05:16:26.875720 29389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:26.876078 29389 log.cc:824] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:26.876978 29389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446: No bootstrap required, opened a new log
I20260812 05:16:26.877964 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:26.875638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.18.166.254:45607)
0812 05:16:26.875850 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:26.875860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:26.875897 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:16:26.876440 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:26.876449 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:26.876506 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:26.876671 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:26.877474 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:26.877482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:26.877482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:26.877487 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:26.877534 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:26.877545 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:26.877626 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:26.877701 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 05:16:26.877771 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260812 05:16:26.878945 29389 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5064f778c5e49e29a10881a364e5446" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 45607 } }
I20260812 05:16:26.879117 29389 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:26.879166 29389 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5064f778c5e49e29a10881a364e5446, State: Initialized, Role: FOLLOWER
I20260812 05:16:26.879293 29389 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5064f778c5e49e29a10881a364e5446" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 45607 } }
I20260812 05:16:26.879374 29389 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:26.879419 29389 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:26.879469 29389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:26.880429 29389 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5064f778c5e49e29a10881a364e5446" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 45607 } }
I20260812 05:16:26.880563 29389 leader_election.cc:302] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5064f778c5e49e29a10881a364e5446; no voters:
I20260812 05:16:26.880769 29389 leader_election.cc:288] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:16:26.880832 29395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:26.881081 29395 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [term 1 LEADER]: Becoming Leader. State: Replica: c5064f778c5e49e29a10881a364e5446, State: Running, Role: LEADER
I20260812 05:16:26.881099 29389 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:26.881249 29395 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5064f778c5e49e29a10881a364e5446" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 45607 } }
I20260812 05:16:26.881705 29397 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5064f778c5e49e29a10881a364e5446. Latest consensus state: current_term: 1 leader_uuid: "c5064f778c5e49e29a10881a364e5446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5064f778c5e49e29a10881a364e5446" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 45607 } } }
I20260812 05:16:26.881701 29396 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5064f778c5e49e29a10881a364e5446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5064f778c5e49e29a10881a364e5446" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 45607 } } }
I20260812 05:16:26.881812 29397 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:26.881840 29396 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:26.882457 29403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:26.883031 29403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:16:26.884891 29403 catalog_manager.cc:1347] Generated new cluster ID: 6ed9b56b062540c5a946be1e9d519a2a
I20260812 05:16:26.884963 29403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:26.896123 29403 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:16:26.896353 29403 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:26.919788 29403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5064f778c5e49e29a10881a364e5446: Generated new TSK 0
I20260812 05:16:26.920043 29403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:16:26.926617 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:45607
--builtin_ntp_servers=127.18.166.212:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 05:16:27.014932 29414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:27.015174 29414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:27.015202 29414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:27.017067 29414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:27.017129 29414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:27.017163 29414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:27.018749 29414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:45607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:27.019049 29414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:27.019384 29414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:27.024703 29420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:27.024698 29419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:27.025218 29422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:27.025377 29414 server_base.cc:1034] running on GCE node
I20260812 05:16:27.025645 29414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:27.025877 29414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:27.027026 29414 hybrid_clock.cc:648] HybridClock initialized: now 1786511787027011 us; error 35 us; skew 500 ppm
I20260812 05:16:27.028447 29414 webserver.cc:460] Webserver started at http://127.18.166.193:41893/ using document root <none> and password file <none>
I20260812 05:16:27.028725 29414 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:27.028798 29414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:27.028955 29414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:27.030201 29414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "3f71935563d24297ba505b736b8c4838"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "ce95f6650b510cc2903a7ac7eb20b02d"
server_key_iv: "1bf6eabc0a1da041b007c2b3fb3c2b01"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.030603 29414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "3f71935563d24297ba505b736b8c4838"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "ce95f6650b510cc2903a7ac7eb20b02d"
server_key_iv: "1bf6eabc0a1da041b007c2b3fb3c2b01"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.032145 29414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:16:27.033033 29429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.033228 29414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:27.033344 29414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "3f71935563d24297ba505b736b8c4838"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "ce95f6650b510cc2903a7ac7eb20b02d"
server_key_iv: "1bf6eabc0a1da041b007c2b3fb3c2b01"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.033453 29414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:27.037194 29414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:27.037575 29414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:27.037739 29414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:27.037971 29414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:27.038316 29414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:27.038371 29414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.038414 29414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:27.038442 29414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.044800 29414 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:36939
I20260812 05:16:27.044864 29551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:36939 every 8 connection(s)
I20260812 05:16:27.045258 29414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:27.048586 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:27.046118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:37289 (local address 127.18.166.254:45607)
0812 05:16:27.046213 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:27.046216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:27.046481 (+ 265us) server_negotiation.cc:407] Connection header received
0812 05:16:27.046983 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:27.046984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:27.046990 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:27.047048 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:27.048327 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:27.048329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:27.048329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:27.048335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:27.048352 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:27.048357 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:27.048421 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:27.048507 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:16:27.048521 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 05:16:27.048812 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:27.046108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:45607 (local address 127.18.166.193:37289)
0812 05:16:27.046349 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:27.046388 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:27.046897 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:27.047102 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:27.047106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:27.047115 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:27.048111 (+ 996us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:16:27.048112 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:27.048146 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:16:27.048179 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:27.048428 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:27.048434 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:27.048437 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:27.048500 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 05:16:27.048558 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":133,"threads_started":1}
I20260812 05:16:27.049389 29552 heartbeater.cc:347] Connected to a master server at 127.18.166.254:45607
I20260812 05:16:27.049528 29552 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:27.049670 29552 heartbeater.cc:511] Master 127.18.166.254:45607 requested a full tablet report, sending...
I20260812 05:16:27.050258 29352 ts_manager.cc:194] Registered new tserver with Master: 3f71935563d24297ba505b736b8c4838 (127.18.166.193:36939)
I20260812 05:16:27.052702 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29414
I20260812 05:16:27.052806 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:16:27.053041 19099 external_mini_cluster.cc:1428] Setting key e4bfdc4f217b26e8ba1050edc10a9a07
I20260812 05:16:27.054105 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:45607
--builtin_ntp_servers=127.18.166.212:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 05:16:27.141593 29555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:27.141857 29555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:27.141901 29555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:27.143239 29555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:27.143316 29555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:27.143365 29555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:27.144889 29555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:45607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:27.145229 29555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:27.145622 29555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:27.151132 29564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:27.151151 29561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:27.151160 29560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:27.151382 29555 server_base.cc:1034] running on GCE node
I20260812 05:16:27.151670 29555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:27.151963 29555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:27.153146 29555 hybrid_clock.cc:648] HybridClock initialized: now 1786511787153122 us; error 41 us; skew 500 ppm
I20260812 05:16:27.154649 29555 webserver.cc:460] Webserver started at http://127.18.166.194:36945/ using document root <none> and password file <none>
I20260812 05:16:27.154923 29555 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:27.154994 29555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:27.155159 29555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:27.156335 29555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "4660a94f50694dc18d548144c4302ba6"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "50b9156a008a4c2c336628a18f011ac1"
server_key_iv: "cfc47c6f126daf801206dccacd527afc"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.156742 29555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "4660a94f50694dc18d548144c4302ba6"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "50b9156a008a4c2c336628a18f011ac1"
server_key_iv: "cfc47c6f126daf801206dccacd527afc"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.158376 29555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:16:27.159206 29570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.159423 29555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:27.159534 29555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "4660a94f50694dc18d548144c4302ba6"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "50b9156a008a4c2c336628a18f011ac1"
server_key_iv: "cfc47c6f126daf801206dccacd527afc"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.159634 29555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:27.163412 29555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:27.163735 29555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:27.163883 29555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:27.164180 29555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:27.164527 29555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:27.164580 29555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.164633 29555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:27.164662 29555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.170930 29555 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:36377
I20260812 05:16:27.171005 29687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:36377 every 8 connection(s)
I20260812 05:16:27.171377 29555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:27.174757 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:27.172222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:48997 (local address 127.18.166.254:45607)
0812 05:16:27.172294 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:27.172297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:27.172569 (+ 272us) server_negotiation.cc:407] Connection header received
0812 05:16:27.173177 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:27.173179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:27.173185 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:27.173253 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:27.174539 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:27.174541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:27.174541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:27.174545 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:27.174561 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:27.174565 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:27.174624 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:27.174674 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 05:16:27.174688 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:16:27.175002 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:27.172222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:45607 (local address 127.18.166.194:48997)
0812 05:16:27.172428 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:27.172465 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:27.172980 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:27.173315 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:27.173319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:27.173328 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:27.174383 (+ 1055us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:16:27.174383 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:27.174416 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:16:27.174450 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:27.174623 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:27.174626 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:27.174629 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:27.174685 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:16:27.174741 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260812 05:16:27.175570 29688 heartbeater.cc:347] Connected to a master server at 127.18.166.254:45607
I20260812 05:16:27.175671 29688 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:27.175800 29688 heartbeater.cc:511] Master 127.18.166.254:45607 requested a full tablet report, sending...
I20260812 05:16:27.176189 29352 ts_manager.cc:194] Registered new tserver with Master: 4660a94f50694dc18d548144c4302ba6 (127.18.166.194:36377)
I20260812 05:16:27.179942 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29555
I20260812 05:16:27.180089 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:16:27.180295 19099 external_mini_cluster.cc:1428] Setting key 7a933f402aa06606194c028ba52b30eb
I20260812 05:16:27.181468 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:45607
--builtin_ntp_servers=127.18.166.212:45873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 05:16:27.267381 29691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:27.267633 29691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:27.267676 29691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:27.268869 29691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:27.268934 29691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:27.268981 29691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:27.270448 29691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:45873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:45607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:27.270722 29691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:27.271026 29691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:27.276577 29697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:27.276547 29696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:27.276665 29691 server_base.cc:1034] running on GCE node
W20260812 05:16:27.276547 29699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:27.277119 29691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:27.277387 29691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:27.278622 29691 hybrid_clock.cc:648] HybridClock initialized: now 1786511787278605 us; error 43 us; skew 500 ppm
I20260812 05:16:27.280133 29691 webserver.cc:460] Webserver started at http://127.18.166.195:46709/ using document root <none> and password file <none>
I20260812 05:16:27.280439 29691 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:27.280498 29691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:27.280723 29691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:27.282086 29691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "0888683ae398484f9ce8b1e4274f1a47"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "43c738eda65c34ede392af53d3ed325b"
server_key_iv: "db8d5458eb3f11257d52bc6238413112"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.282521 29691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "0888683ae398484f9ce8b1e4274f1a47"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "43c738eda65c34ede392af53d3ed325b"
server_key_iv: "db8d5458eb3f11257d52bc6238413112"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.284034 29691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:16:27.284945 29705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.285254 29691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:27.285354 29691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "0888683ae398484f9ce8b1e4274f1a47"
format_stamp: "Formatted at 2026-08-12 05:16:27 on dist-test-slave-srgv"
server_key: "43c738eda65c34ede392af53d3ed325b"
server_key_iv: "db8d5458eb3f11257d52bc6238413112"
server_key_version: "encryptionkey@0"
I20260812 05:16:27.285470 29691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:27.289345 29691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:27.289726 29691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:27.289897 29691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:27.290151 29691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:27.290539 29691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:27.290584 29691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.290608 29691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:27.290658 29691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:27.297462 29691 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:44931
I20260812 05:16:27.297573 29823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:44931 every 8 connection(s)
I20260812 05:16:27.297981 29691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:27.301502 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:27.298950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:47063 (local address 127.18.166.254:45607)
0812 05:16:27.299052 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:27.299055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:27.299265 (+ 210us) server_negotiation.cc:407] Connection header received
0812 05:16:27.299805 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:27.299807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:27.299813 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:27.299873 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:27.301157 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:27.301158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:27.301159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:16:27.301163 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:27.301182 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:16:27.301186 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:27.301260 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:27.301346 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:16:27.301361 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 05:16:27.301720 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:27.298873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:45607 (local address 127.18.166.195:47063)
0812 05:16:27.299131 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:27.299171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:27.299708 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:27.299933 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:27.299937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:27.299946 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:27.300970 (+ 1024us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:16:27.300971 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:27.301005 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:16:27.301040 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:27.301274 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:27.301280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:27.301283 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:16:27.301352 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:16:27.301467 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":118,"threads_started":1}
I20260812 05:16:27.302372 29824 heartbeater.cc:347] Connected to a master server at 127.18.166.254:45607
I20260812 05:16:27.302493 29824 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:27.302667 29824 heartbeater.cc:511] Master 127.18.166.254:45607 requested a full tablet report, sending...
I20260812 05:16:27.303084 29352 ts_manager.cc:194] Registered new tserver with Master: 0888683ae398484f9ce8b1e4274f1a47 (127.18.166.195:44931)
I20260812 05:16:27.307482 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29691
I20260812 05:16:27.307586 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:16:27.307819 19099 external_mini_cluster.cc:1428] Setting key 69ed12c78c761ec7c9b88579f9c71871
I20260812 05:16:27.308876 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 05:16:27.311084 29393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:16:27.310426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.18.166.254:45607)
0812 05:16:27.310497 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:27.310500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:27.310638 (+ 138us) server_negotiation.cc:407] Connection header received
0812 05:16:27.310695 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:27.310696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:27.310703 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:27.310754 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:27.310997 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47156: BlockingRecv error: recv got EOF from 127.0.0.1:47156 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:16:27.311144 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29414
I20260812 05:16:27.322292 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29555
I20260812 05:16:27.328431 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29691
I20260812 05:16:27.337172 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29320
2026-08-12T05:16:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (607 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:16:27 dist-test-slave-srgv krb5kdc[29836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:16:27 dist-test-slave-srgv krb5kdc[29836](info): set up 2 sockets
Aug 12 05:16:27 dist-test-slave-srgv krb5kdc[29836](info): commencing operation
krb5kdc: starting...
W20260812 05:16:28.428836 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:16:28 dist-test-slave-srgv krb5kdc[29836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:16:28.558691 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:40789
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:40789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:28.645824 29850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:28.646070 29850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:28.646097 29850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:28.647380 29850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:16:28.647439 29850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:28.647454 29850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:28.647467 29850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:16:28.647480 29850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:16:28.649111 29850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:40789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:40789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:28.649374 29850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:28.649672 29850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:28.655141 29858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:28.655130 29856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:28.655133 29855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:28.655434 29850 server_base.cc:1034] running on GCE node
I20260812 05:16:28.655771 29850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:28.656126 29850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:28.657313 29850 hybrid_clock.cc:648] HybridClock initialized: now 1786511788657295 us; error 57 us; skew 500 ppm
Aug 12 05:16:28 dist-test-slave-srgv krb5kdc[29836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:28.660341 29850 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:16:28.660934 29850 webserver.cc:460] Webserver started at http://127.18.166.254:43851/ using document root <none> and password file <none>
I20260812 05:16:28.661212 29850 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:28.661295 29850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:28.661527 29850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:28.662724 29850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "8813b4fd3f434218b232ca842b70b8b7"
format_stamp: "Formatted at 2026-08-12 05:16:28 on dist-test-slave-srgv"
server_key: "c9389c61e534f4f9abe96a6464318d26"
server_key_iv: "b3c0668396270645ff1c147f8c8d61ab"
server_key_version: "encryptionkey@0"
I20260812 05:16:28.663148 29850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "8813b4fd3f434218b232ca842b70b8b7"
format_stamp: "Formatted at 2026-08-12 05:16:28 on dist-test-slave-srgv"
server_key: "c9389c61e534f4f9abe96a6464318d26"
server_key_iv: "b3c0668396270645ff1c147f8c8d61ab"
server_key_version: "encryptionkey@0"
I20260812 05:16:28.665149 29850 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 05:16:28.666236 29865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:28.666447 29850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:28.666565 29850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "8813b4fd3f434218b232ca842b70b8b7"
format_stamp: "Formatted at 2026-08-12 05:16:28 on dist-test-slave-srgv"
server_key: "c9389c61e534f4f9abe96a6464318d26"
server_key_iv: "b3c0668396270645ff1c147f8c8d61ab"
server_key_version: "encryptionkey@0"
I20260812 05:16:28.666702 29850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:28.711079 29850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:28.713919 29850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:28.714124 29850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:28.720041 29850 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:40789
I20260812 05:16:28.720083 29919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:40789 every 8 connection(s)
I20260812 05:16:28.720543 29850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:16:28.721800 29920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:28.724439 29920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7: Bootstrap starting.
I20260812 05:16:28.725159 29920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:28.725559 29920 log.cc:824] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:28.725628 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29850
I20260812 05:16:28.725770 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:16:28.726004 19099 external_mini_cluster.cc:1428] Setting key e312b64bcf1eded381c3404e4e1ba70c
I20260812 05:16:28.726758 29920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7: No bootstrap required, opened a new log
I20260812 05:16:28.729091 29920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8813b4fd3f434218b232ca842b70b8b7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40789 } }
I20260812 05:16:28.729302 29920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:28.729351 29920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8813b4fd3f434218b232ca842b70b8b7, State: Initialized, Role: FOLLOWER
I20260812 05:16:28.729550 29920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8813b4fd3f434218b232ca842b70b8b7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40789 } }
I20260812 05:16:28.729660 29920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:16:28.729717 29920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:16:28.729765 29920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:28.730643 29920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8813b4fd3f434218b232ca842b70b8b7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40789 } }
I20260812 05:16:28.730775 29920 leader_election.cc:302] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8813b4fd3f434218b232ca842b70b8b7; no voters:
I20260812 05:16:28.730960 29920 leader_election.cc:288] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:16:28.731040 29926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:28.731208 29926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [term 1 LEADER]: Becoming Leader. State: Replica: 8813b4fd3f434218b232ca842b70b8b7, State: Running, Role: LEADER
I20260812 05:16:28.731245 29920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:16:28.731410 29926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8813b4fd3f434218b232ca842b70b8b7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40789 } }
I20260812 05:16:28.731887 29926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8813b4fd3f434218b232ca842b70b8b7. Latest consensus state: current_term: 1 leader_uuid: "8813b4fd3f434218b232ca842b70b8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8813b4fd3f434218b232ca842b70b8b7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40789 } } }
I20260812 05:16:28.732066 29926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:28.733047 29935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:16:28.733059 29928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8813b4fd3f434218b232ca842b70b8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8813b4fd3f434218b232ca842b70b8b7" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40789 } } }
I20260812 05:16:28.733275 29928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:16:28.734175 29935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:16:28 dist-test-slave-srgv krb5kdc[29836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:28.740144 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:28.726623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.18.166.254:40789)
0812 05:16:28.726961 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:28.726972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:28.727010 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:16:28.728214 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:28.728225 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:28.728300 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:28.728482 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:28.729150 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:28.730970 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:28.732277 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:28.733136 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:28.733267 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:28.736130 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:28.736135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:28.736136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:28.736149 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:28.738318 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:28.738843 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:28.738847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:28.738847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:28.738942 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:28.739172 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:28.739173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:28.739173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:28.739451 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:28.739583 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:28.739746 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:16:28.739867 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":166,"threads_started":1}
I20260812 05:16:28.746429 29935 catalog_manager.cc:1347] Generated new cluster ID: 34a1a4745a0443d7b2eeed47c60a51ba
I20260812 05:16:28.746520 29935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:16:28.766489 29935 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:16:28.767135 29935 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:16:28.774085 29935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7: Generated new TSK 0
I20260812 05:16:28.774329 29935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:16:28.816793 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:40789
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:28.932555 29947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:28.932821 29947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:28.932850 29947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:28.934231 29947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:28.934290 29947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:28.934322 29947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:16:28.935900 29947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:40789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:28.936132 29947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:28.936344 29947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:28.942066 29953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:28.942076 29947 server_base.cc:1034] running on GCE node
W20260812 05:16:28.942095 29955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:28.942083 29952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:28.942535 29947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:28.942853 29947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:28.944042 29947 hybrid_clock.cc:648] HybridClock initialized: now 1786511788944028 us; error 49 us; skew 500 ppm
Aug 12 05:16:28 dist-test-slave-srgv krb5kdc[29836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:28.947018 29947 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:16:28.947505 29947 webserver.cc:460] Webserver started at http://127.18.166.193:43101/ using document root <none> and password file <none>
I20260812 05:16:28.947769 29947 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:28.947820 29947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:28.947983 29947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:28.949172 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "60ca8b3f4891481ca4b8d714bc10d91c"
format_stamp: "Formatted at 2026-08-12 05:16:28 on dist-test-slave-srgv"
server_key: "2def60b827495016e166d1e21ab1dc48"
server_key_iv: "4b79f4ad557fd9a41e09b0a3189b2141"
server_key_version: "encryptionkey@0"
I20260812 05:16:28.949633 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "60ca8b3f4891481ca4b8d714bc10d91c"
format_stamp: "Formatted at 2026-08-12 05:16:28 on dist-test-slave-srgv"
server_key: "2def60b827495016e166d1e21ab1dc48"
server_key_iv: "4b79f4ad557fd9a41e09b0a3189b2141"
server_key_version: "encryptionkey@0"
I20260812 05:16:28.951205 29947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:28.952085 29963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:28.952315 29947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:28.952435 29947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "60ca8b3f4891481ca4b8d714bc10d91c"
format_stamp: "Formatted at 2026-08-12 05:16:28 on dist-test-slave-srgv"
server_key: "2def60b827495016e166d1e21ab1dc48"
server_key_iv: "4b79f4ad557fd9a41e09b0a3189b2141"
server_key_version: "encryptionkey@0"
I20260812 05:16:28.952524 29947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:28.962453 29947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:28.964890 29947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:28.965083 29947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:28.965355 29947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:28.965790 29947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:28.965849 29947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:28.965894 29947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:28.965922 29947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:28.972873 29947 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:35719
I20260812 05:16:28.972930 30085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:35719 every 8 connection(s)
I20260812 05:16:28.973306 29947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:16:28.974380 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 29947
I20260812 05:16:28.974488 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:16:28.974668 19099 external_mini_cluster.cc:1428] Setting key 07c54a920d637a3ccb4cfbc8309bf662
Aug 12 05:16:28 dist-test-slave-srgv krb5kdc[29836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:16:28.984516 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:28.974150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:48351 (local address 127.18.166.254:40789)
0812 05:16:28.974831 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:28.974833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:28.974846 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:16:28.975126 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:28.975127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:28.975139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:28.975225 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:28.976335 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:28.976897 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:28.978187 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:28.978411 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:28.978501 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:28.980976 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:28.980978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:28.980978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:28.980988 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:28.982710 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:28.983302 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:28.983304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:28.983304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:28.983367 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:28.983686 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:28.983687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:28.983688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:28.983829 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:28.983915 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:28.984252 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 05:16:28.984364 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653}
I20260812 05:16:28.984688 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:28.974100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:40789 (local address 127.18.166.193:48351)
0812 05:16:28.974316 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:28.974363 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:28.975012 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:28.975309 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:28.975314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:28.975977 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:28.976213 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:28.976219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:28.977030 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:28.977031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:28.978068 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:28.978072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:28.978518 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:28.978519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:28.978668 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:28.979300 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:28.979310 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:28.980842 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:28.982840 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:28.982854 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:28.982857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:28.983151 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:28.983444 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:28.983445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:28.983446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:28.983621 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:28.983903 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:28.983912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:28.984096 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 05:16:28.984228 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 05:16:28.984420 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
I20260812 05:16:28.985281 30086 heartbeater.cc:347] Connected to a master server at 127.18.166.254:40789
I20260812 05:16:28.985409 30086 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:28.985672 30086 heartbeater.cc:511] Master 127.18.166.254:40789 requested a full tablet report, sending...
I20260812 05:16:28.986340 29883 ts_manager.cc:194] Registered new tserver with Master: 60ca8b3f4891481ca4b8d714bc10d91c (127.18.166.193:35719)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
I20260812 05:16:28.987138 29883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:48351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:16:29.005398 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:40789
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:29.094480 30091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:29.094856 30091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:29.094901 30091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:29.097136 30091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:29.097239 30091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:29.097309 30091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:16:29.099107 30091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:40789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:29.099373 30091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:29.099584 30091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:29.105046 30097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:29.105046 30100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:29.105050 30098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:29.105485 30091 server_base.cc:1034] running on GCE node
I20260812 05:16:29.105769 30091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:29.106062 30091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:29.107270 30091 hybrid_clock.cc:648] HybridClock initialized: now 1786511789107261 us; error 44 us; skew 500 ppm
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[29836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511789, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:29.110255 30091 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:16:29.110733 30091 webserver.cc:460] Webserver started at http://127.18.166.194:34229/ using document root <none> and password file <none>
I20260812 05:16:29.111017 30091 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:29.111109 30091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:29.111258 30091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:29.112385 30091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "e13e3b67a60746f786877187e4a84bdc"
format_stamp: "Formatted at 2026-08-12 05:16:29 on dist-test-slave-srgv"
server_key: "0c151738333d1cd290bf638d3c45c28a"
server_key_iv: "cf0a038dfe0e6f47391eb217ea3cdd13"
server_key_version: "encryptionkey@0"
I20260812 05:16:29.112718 30091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "e13e3b67a60746f786877187e4a84bdc"
format_stamp: "Formatted at 2026-08-12 05:16:29 on dist-test-slave-srgv"
server_key: "0c151738333d1cd290bf638d3c45c28a"
server_key_iv: "cf0a038dfe0e6f47391eb217ea3cdd13"
server_key_version: "encryptionkey@0"
I20260812 05:16:29.114315 30091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260812 05:16:29.115139 30107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:29.115362 30091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:16:29.115474 30091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "e13e3b67a60746f786877187e4a84bdc"
format_stamp: "Formatted at 2026-08-12 05:16:29 on dist-test-slave-srgv"
server_key: "0c151738333d1cd290bf638d3c45c28a"
server_key_iv: "cf0a038dfe0e6f47391eb217ea3cdd13"
server_key_version: "encryptionkey@0"
I20260812 05:16:29.115558 30091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:29.127835 30091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:29.130455 30091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:29.130636 30091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:29.130945 30091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:29.131311 30091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:29.131441 30091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:29.131502 30091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:29.131524 30091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:29.138144 30091 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:32969
I20260812 05:16:29.138233 30227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:32969 every 8 connection(s)
I20260812 05:16:29.138588 30091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:16:29.141754 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 30091
I20260812 05:16:29.141897 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:16:29.142125 19099 external_mini_cluster.cc:1428] Setting key 263f3d12191736f8ba9549a7166fe8a0
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[29836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511789, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:29.150960 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.139549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:42181 (local address 127.18.166.254:40789)
0812 05:16:29.139748 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.139751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.139764 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:16:29.140359 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.140361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.140374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.140432 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:29.141551 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.142192 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.143449 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.143680 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.143766 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.146774 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:29.146776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:29.146776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:29.146789 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:29.148875 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:29.149605 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:29.149606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:29.149607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:29.149697 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:29.150058 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:29.150060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:29.150060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:29.150269 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:29.150352 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.150708 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.150821 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260812 05:16:29.151011 30230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.139440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:40789 (local address 127.18.166.194:42181)
0812 05:16:29.139627 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.139677 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.140232 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.140530 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.140534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.141182 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:29.141397 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.141402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.142311 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.142312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.143344 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.143347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.143799 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.143800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.143948 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.144668 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:29.144681 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:29.146632 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:29.149031 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:29.149046 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:29.149049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:29.149370 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:29.149803 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:29.149804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:29.149804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:29.149955 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:29.150353 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:29.150360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:29.150529 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.150624 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.150805 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260812 05:16:29.151708 30228 heartbeater.cc:347] Connected to a master server at 127.18.166.254:40789
I20260812 05:16:29.151860 30228 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:29.152132 30228 heartbeater.cc:511] Master 127.18.166.254:40789 requested a full tablet report, sending...
I20260812 05:16:29.152670 29883 ts_manager.cc:194] Registered new tserver with Master: e13e3b67a60746f786877187e4a84bdc (127.18.166.194:32969)
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:16:29.153148 29883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:42181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:16:29.173254 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:40789
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:16:29.261947 30233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:16:29.262216 30233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:16:29.262246 30233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:16:29.263582 30233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:16:29.263634 30233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:16:29.263665 30233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:16:29.265385 30233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:40789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:16:29.265776 30233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:16:29.266007 30233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:16:29.271941 30241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:16:29.271965 30243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:29.272248 30233 server_base.cc:1034] running on GCE node
W20260812 05:16:29.271941 30239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:16:29.272573 30233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:16:29.272886 30233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:16:29.274067 30233 hybrid_clock.cc:648] HybridClock initialized: now 1786511789274046 us; error 58 us; skew 500 ppm
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[29836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511789, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:16:29.276937 30233 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:16:29.277623 30233 webserver.cc:460] Webserver started at http://127.18.166.195:33355/ using document root <none> and password file <none>
I20260812 05:16:29.277889 30233 fs_manager.cc:359] Metadata directory not provided
I20260812 05:16:29.277963 30233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:16:29.278137 30233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:16:29.279444 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "2d9ef4a3af37400a9f293a03b4d0580b"
format_stamp: "Formatted at 2026-08-12 05:16:29 on dist-test-slave-srgv"
server_key: "bb0350feca0ea740b2c2d4aa844b9369"
server_key_iv: "bfad775a23ac946ce11e7d115fff6897"
server_key_version: "encryptionkey@0"
I20260812 05:16:29.279865 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "2d9ef4a3af37400a9f293a03b4d0580b"
format_stamp: "Formatted at 2026-08-12 05:16:29 on dist-test-slave-srgv"
server_key: "bb0350feca0ea740b2c2d4aa844b9369"
server_key_iv: "bfad775a23ac946ce11e7d115fff6897"
server_key_version: "encryptionkey@0"
I20260812 05:16:29.281396 30233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:16:29.282401 30250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:29.282614 30233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:16:29.282742 30233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "2d9ef4a3af37400a9f293a03b4d0580b"
format_stamp: "Formatted at 2026-08-12 05:16:29 on dist-test-slave-srgv"
server_key: "bb0350feca0ea740b2c2d4aa844b9369"
server_key_iv: "bfad775a23ac946ce11e7d115fff6897"
server_key_version: "encryptionkey@0"
I20260812 05:16:29.282831 30233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:16:29.329210 30233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:16:29.331674 30233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:16:29.331872 30233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:16:29.332194 30233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:16:29.332551 30233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:16:29.332607 30233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:29.332651 30233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:16:29.332680 30233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:16:29.339355 30233 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43435
I20260812 05:16:29.339396 30375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43435 every 8 connection(s)
I20260812 05:16:29.339772 30233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:16:29.340099 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 30233
I20260812 05:16:29.340201 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:16:29.340408 19099 external_mini_cluster.cc:1428] Setting key 91297ad4e0248d6a98e8fe80ae61b943
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[29836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511789, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:16:29.350381 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.340866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:48011 (local address 127.18.166.254:40789)
0812 05:16:29.340915 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.340917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.341160 (+ 243us) server_negotiation.cc:407] Connection header received
0812 05:16:29.341772 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.341773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.341785 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.341827 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:29.342851 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.343376 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.344423 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.344638 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.344723 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.347144 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:29.347145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:29.347146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:29.347154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:29.348777 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:29.349281 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:29.349282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:29.349282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:29.349347 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:29.349713 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:29.349714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:29.349714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:29.349842 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:29.349902 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.350204 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.350295 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:16:29.350485 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.340850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:40789 (local address 127.18.166.195:48011)
0812 05:16:29.341015 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.341060 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.341661 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.341912 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.341916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.342523 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:16:29.342746 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.342751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.343489 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.343492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.344323 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.344325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.344739 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.344740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.344870 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.345494 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:16:29.345503 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:16:29.347021 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:16:29.348898 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:29.348907 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:29.348910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:29.349196 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:29.349502 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:16:29.349503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:16:29.349503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:16:29.349639 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:16:29.349891 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:16:29.349895 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:16:29.350036 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.350116 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.350293 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260812 05:16:29.351111 30376 heartbeater.cc:347] Connected to a master server at 127.18.166.254:40789
I20260812 05:16:29.351212 30376 heartbeater.cc:464] Registering TS with master...
I20260812 05:16:29.351395 30376 heartbeater.cc:511] Master 127.18.166.254:40789 requested a full tablet report, sending...
I20260812 05:16:29.351871 29883 ts_manager.cc:194] Registered new tserver with Master: 2d9ef4a3af37400a9f293a03b4d0580b (127.18.166.195:43435)
I20260812 05:16:29.352356 29883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:48011
I20260812 05:16:29.352347 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:16:29.360345 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.353901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.18.166.254:40789)
0812 05:16:29.353945 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.353947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.354063 (+ 116us) server_negotiation.cc:407] Connection header received
0812 05:16:29.354106 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.354107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.354116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.354166 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:16:29.354716 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.355192 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.356017 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.356205 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.356275 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.357683 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:16:29.357685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:16:29.357685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:16:29.357692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:16:29.359211 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:29.359600 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:29.359601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:29.359601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:29.359659 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:16:29.359833 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:16:29.359833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:16:29.359833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:16:29.359991 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:16:29.360073 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.360177 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.360273 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:16:29.362181 29883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:16:29.363921 29883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:16:29.371693 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.366898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.18.166.193:35719)
0812 05:16:29.367382 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.367387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.367396 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:29.367435 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.367437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.367454 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.367692 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:29.367773 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.368489 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.370165 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.371160 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.371257 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.371317 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.371529 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.371586 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":96,"threads_started":1}
I20260812 05:16:29.371693 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.366688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35719 (local address 127.0.0.1:41432)
0812 05:16:29.367149 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.367164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.367228 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.367566 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.367567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.367568 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:29.367613 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.367614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.368615 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.368617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.370050 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.370055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.371257 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.371258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.371376 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.371394 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.371509 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.371563 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":3,"thread_start_us":122,"threads_started":1}
I20260812 05:16:29.371708 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.366682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43435 (local address 127.0.0.1:49446)
0812 05:16:29.366960 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.366969 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.367008 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.367743 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.367743 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.367745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:29.367780 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.367781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.369049 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.369050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.370161 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.370163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.371326 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.371327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.371394 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.371398 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.371478 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.371556 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260812 05:16:29.372141 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.366883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.18.166.195:43435)
0812 05:16:29.367340 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.367344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.367354 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:16:29.367641 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.367644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.367663 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.367757 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:29.367830 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.368899 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.370250 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.371145 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.371226 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.371287 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.372021 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.372076 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":83,"threads_started":1}
I20260812 05:16:29.373028 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.366682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:32969 (local address 127.0.0.1:46014)
0812 05:16:29.366883 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.366927 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.367007 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.367504 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.367507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.367510 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:29.367610 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.367612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.370407 (+ 2795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.370408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.371480 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.371483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.372756 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.372757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.372829 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.372834 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.372890 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.372947 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"spinlock_wait_cycles":896,"thread_start_us":106,"threads_started":1}
I20260812 05:16:29.373075 30303 tablet_service.cc:1467] Processing CreateTablet for tablet 79fd928048834b0c868d3c96b75eec81 (DEFAULT_TABLE table=test-table [id=b8b752e69ee64ecbb7c65e7239533566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:29.373292 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.367094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.18.166.194:32969)
0812 05:16:29.367289 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.367296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.367309 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:16:29.367359 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.367362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.367381 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.367485 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:29.367677 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.368887 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.371591 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.372652 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.372757 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.372820 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.373152 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.373204 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260812 05:16:29.373322 30014 tablet_service.cc:1467] Processing CreateTablet for tablet 79fd928048834b0c868d3c96b75eec81 (DEFAULT_TABLE table=test-table [id=b8b752e69ee64ecbb7c65e7239533566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:29.373596 30303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79fd928048834b0c868d3c96b75eec81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:29.373729 30014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79fd928048834b0c868d3c96b75eec81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:29.374028 30159 tablet_service.cc:1467] Processing CreateTablet for tablet 79fd928048834b0c868d3c96b75eec81 (DEFAULT_TABLE table=test-table [id=b8b752e69ee64ecbb7c65e7239533566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:16:29.374373 30159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79fd928048834b0c868d3c96b75eec81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:16:29.376546 30393 tablet_bootstrap.cc:492] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: Bootstrap starting.
I20260812 05:16:29.377507 30394 tablet_bootstrap.cc:492] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Bootstrap starting.
I20260812 05:16:29.377668 30393 tablet_bootstrap.cc:654] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:29.377991 30395 tablet_bootstrap.cc:492] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Bootstrap starting.
I20260812 05:16:29.378289 30394 tablet_bootstrap.cc:654] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:29.378278 30393 log.cc:824] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:29.378808 30395 tablet_bootstrap.cc:654] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Neither blocks nor log segments found. Creating new log.
I20260812 05:16:29.378861 30394 log.cc:824] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:29.379410 30393 tablet_bootstrap.cc:492] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: No bootstrap required, opened a new log
I20260812 05:16:29.379498 30393 ts_tablet_manager.cc:1397] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:16:29.379503 30395 log.cc:824] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Log is configured to *not* fsync() on all Append() calls
I20260812 05:16:29.379726 30394 tablet_bootstrap.cc:492] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: No bootstrap required, opened a new log
I20260812 05:16:29.379799 30394 ts_tablet_manager.cc:1397] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:16:29.380287 30395 tablet_bootstrap.cc:492] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: No bootstrap required, opened a new log
I20260812 05:16:29.380379 30395 ts_tablet_manager.cc:1397] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:16:29.381744 30393 raft_consensus.cc:348] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.381978 30393 raft_consensus.cc:374] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:29.382025 30393 raft_consensus.cc:729] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ca8b3f4891481ca4b8d714bc10d91c, State: Initialized, Role: FOLLOWER
I20260812 05:16:29.382030 30394 raft_consensus.cc:348] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.382175 30393 consensus_queue.cc:260] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.382241 30394 raft_consensus.cc:374] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:29.382278 30394 raft_consensus.cc:729] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9ef4a3af37400a9f293a03b4d0580b, State: Initialized, Role: FOLLOWER
I20260812 05:16:29.382424 30394 consensus_queue.cc:260] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.382526 30393 ts_tablet_manager.cc:1428] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:16:29.382650 30086 heartbeater.cc:503] Master 127.18.166.254:40789 was elected leader, sending a full tablet report...
I20260812 05:16:29.382567 30395 raft_consensus.cc:348] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.382707 30394 ts_tablet_manager.cc:1428] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 05:16:29.382735 30395 raft_consensus.cc:374] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:16:29.382771 30395 raft_consensus.cc:729] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13e3b67a60746f786877187e4a84bdc, State: Initialized, Role: FOLLOWER
I20260812 05:16:29.382853 30376 heartbeater.cc:503] Master 127.18.166.254:40789 was elected leader, sending a full tablet report...
I20260812 05:16:29.382922 30395 consensus_queue.cc:260] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.383232 30395 ts_tablet_manager.cc:1428] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:16:29.383234 30228 heartbeater.cc:503] Master 127.18.166.254:40789 was elected leader, sending a full tablet report...
W20260812 05:16:29.389283 30229 tablet.cc:2369] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:16:29.474263 30087 tablet.cc:2369] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:16:29.488756 30401 raft_consensus.cc:482] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:16:29.488899 30401 raft_consensus.cc:504] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.489349 30401 leader_election.cc:288] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d9ef4a3af37400a9f293a03b4d0580b (127.18.166.195:43435), e13e3b67a60746f786877187e4a84bdc (127.18.166.194:32969)
I20260812 05:16:29.492591 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.489765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:45361 (local address 127.18.166.195:43435)
0812 05:16:29.489812 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.489814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.489952 (+ 138us) server_negotiation.cc:407] Connection header received
0812 05:16:29.490030 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.490031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.490048 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.490110 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:29.490275 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.490756 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.491796 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.492191 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.492302 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.492312 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.492473 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.492515 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:16:29.492616 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.489717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43435 (local address 127.18.166.193:45361)
0812 05:16:29.489851 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.489870 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.489962 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.490114 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.490115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.490116 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:29.490196 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.490198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.490849 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.490850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.491677 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.491679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.492304 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.492306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.492391 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.492395 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.492465 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.492513 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260812 05:16:29.492591 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.489704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:38763 (local address 127.18.166.194:32969)
0812 05:16:29.489804 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.489807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.489892 (+ 85us) server_negotiation.cc:407] Connection header received
0812 05:16:29.489970 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.489971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.489989 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.490055 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:16:29.490234 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.490736 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.491788 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.492190 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.492285 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.492296 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.492465 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.492506 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 05:16:29.492619 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.489594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:32969 (local address 127.18.166.193:38763)
0812 05:16:29.489801 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.489812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:16:29.489908 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:16:29.490060 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:16:29.490061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:16:29.490063 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:16:29.490154 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.490156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.490841 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.490842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.491674 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:16:29.491677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.492290 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.492292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:16:29.492374 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.492378 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:16:29.492453 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:16:29.492506 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":108,"threads_started":1}
I20260812 05:16:29.493043 30180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79fd928048834b0c868d3c96b75eec81" candidate_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13e3b67a60746f786877187e4a84bdc" is_pre_election: true
I20260812 05:16:29.493037 30326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79fd928048834b0c868d3c96b75eec81" candidate_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" is_pre_election: true
I20260812 05:16:29.493233 30326 raft_consensus.cc:2454] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60ca8b3f4891481ca4b8d714bc10d91c in term 0.
I20260812 05:16:29.493233 30180 raft_consensus.cc:2454] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60ca8b3f4891481ca4b8d714bc10d91c in term 0.
I20260812 05:16:29.493554 29965 leader_election.cc:302] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ca8b3f4891481ca4b8d714bc10d91c, e13e3b67a60746f786877187e4a84bdc; no voters:
I20260812 05:16:29.493708 30401 raft_consensus.cc:2781] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:16:29.493773 30401 raft_consensus.cc:482] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:16:29.493808 30401 raft_consensus.cc:3037] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:29.494620 30401 raft_consensus.cc:504] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.494793 30401 leader_election.cc:288] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [CANDIDATE]: Term 1 election: Requested vote from peers 2d9ef4a3af37400a9f293a03b4d0580b (127.18.166.195:43435), e13e3b67a60746f786877187e4a84bdc (127.18.166.194:32969)
I20260812 05:16:29.494968 30180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79fd928048834b0c868d3c96b75eec81" candidate_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13e3b67a60746f786877187e4a84bdc"
I20260812 05:16:29.494968 30326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79fd928048834b0c868d3c96b75eec81" candidate_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9ef4a3af37400a9f293a03b4d0580b"
I20260812 05:16:29.495062 30326 raft_consensus.cc:3037] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:29.495062 30180 raft_consensus.cc:3037] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:16:29.496043 30180 raft_consensus.cc:2454] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ca8b3f4891481ca4b8d714bc10d91c in term 1.
I20260812 05:16:29.496043 30326 raft_consensus.cc:2454] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ca8b3f4891481ca4b8d714bc10d91c in term 1.
I20260812 05:16:29.496264 29965 leader_election.cc:302] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ca8b3f4891481ca4b8d714bc10d91c, e13e3b67a60746f786877187e4a84bdc; no voters:
I20260812 05:16:29.496379 30401 raft_consensus.cc:2781] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:16:29.496572 30401 raft_consensus.cc:686] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 1 LEADER]: Becoming Leader. State: Replica: 60ca8b3f4891481ca4b8d714bc10d91c, State: Running, Role: LEADER
I20260812 05:16:29.496694 30401 consensus_queue.cc:237] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } }
I20260812 05:16:29.497745 29879 catalog_manager.cc:5495] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c reported cstate change: term changed from 0 to 1, leader changed from <none> to 60ca8b3f4891481ca4b8d714bc10d91c (127.18.166.193). New cstate: current_term: 1 leader_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ca8b3f4891481ca4b8d714bc10d91c" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 } health_report { overall_health: UNKNOWN } } }
I20260812 05:16:29.520025 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.517568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.18.166.193:35719)
0812 05:16:29.517637 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:16:29.517639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:16:29.517648 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:16:29.517690 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:16:29.517691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:16:29.517710 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:16:29.517783 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:16:29.517936 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.518522 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.519260 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:16:29.519483 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:16:29.519552 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:16:29.519735 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:16:29.519841 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:16:29.519907 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:16:29.519934 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 05:16:29.519957 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:16:29.522112 30180 raft_consensus.cc:1261] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 1 FOLLOWER]: Refusing update from remote peer 60ca8b3f4891481ca4b8d714bc10d91c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:29.522128 30326 raft_consensus.cc:1261] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 1 FOLLOWER]: Refusing update from remote peer 60ca8b3f4891481ca4b8d714bc10d91c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:16:29.522562 30407 consensus_queue.cc:1035] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e13e3b67a60746f786877187e4a84bdc" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:29.522732 30401 consensus_queue.cc:1035] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d9ef4a3af37400a9f293a03b4d0580b" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:16:29.525218 30410 mvcc.cc:205] Tried to move back new op lower bound from 7317552289879236608 to 7317552289779224576. Current Snapshot: MvccSnapshot[applied={T|T < 7317552289879236608}]
I20260812 05:16:29.525671 30411 mvcc.cc:205] Tried to move back new op lower bound from 7317552289879236608 to 7317552289779224576. Current Snapshot: MvccSnapshot[applied={T|T < 7317552289879236608}]
I20260812 05:16:29.526095 30412 mvcc.cc:205] Tried to move back new op lower bound from 7317552289879236608 to 7317552289779224576. Current Snapshot: MvccSnapshot[applied={T|T < 7317552289879236608}]
I20260812 05:16:29.527994 29883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:29.528084 29883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:16:29.528893 29883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8813b4fd3f434218b232ca842b70b8b7: Sending DeleteTablet for 3 replicas of tablet 79fd928048834b0c868d3c96b75eec81
I20260812 05:16:29.529256 30014 tablet_service.cc:1514] Processing DeleteTablet for tablet 79fd928048834b0c868d3c96b75eec81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:29 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:41432
I20260812 05:16:29.529273 30159 tablet_service.cc:1514] Processing DeleteTablet for tablet 79fd928048834b0c868d3c96b75eec81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:29 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:46014
I20260812 05:16:29.529525 30422 tablet_replica.cc:330] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c: stopping tablet replica
I20260812 05:16:29.529644 30422 raft_consensus.cc:2229] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:16:29.529795 30422 raft_consensus.cc:2258] T 79fd928048834b0c868d3c96b75eec81 P 60ca8b3f4891481ca4b8d714bc10d91c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:29.529780 30303 tablet_service.cc:1514] Processing DeleteTablet for tablet 79fd928048834b0c868d3c96b75eec81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:16:29 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:49446
I20260812 05:16:29.530032 30423 tablet_replica.cc:330] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: stopping tablet replica
I20260812 05:16:29.530114 30421 tablet_replica.cc:330] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: stopping tablet replica
I20260812 05:16:29.530224 30421 raft_consensus.cc:2229] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:29.530242 30423 raft_consensus.cc:2229] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:16:29.530335 30421 raft_consensus.cc:2258] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:29.530336 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29947
I20260812 05:16:29.530524 30423 raft_consensus.cc:2258] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:16:29.532763 30421 ts_tablet_manager.cc:1905] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:29.534718 30421 ts_tablet_manager.cc:1918] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:29.534842 30421 log.cc:1196] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/79fd928048834b0c868d3c96b75eec81
I20260812 05:16:29.535130 30421 ts_tablet_manager.cc:1939] T 79fd928048834b0c868d3c96b75eec81 P e13e3b67a60746f786877187e4a84bdc: Deleting consensus metadata
I20260812 05:16:29.535652 29867 catalog_manager.cc:4841] TS e13e3b67a60746f786877187e4a84bdc (127.18.166.194:32969): tablet 79fd928048834b0c868d3c96b75eec81 (table test-table [id=b8b752e69ee64ecbb7c65e7239533566]) successfully deleted
W20260812 05:16:29.536649 29868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:35719 (error 108)
I20260812 05:16:29.537009 30423 ts_tablet_manager.cc:1905] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:16:29.537204 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:16:29.537064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35719 (local address 127.0.0.1:41448)
0812 05:16:29.537127 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 05:16:29.537173 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 05:16:29.537330 29868 catalog_manager.cc:4584] TS 60ca8b3f4891481ca4b8d714bc10d91c (127.18.166.193:35719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79fd928048834b0c868d3c96b75eec81: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35719: connect: Connection refused (error 111)
I20260812 05:16:29.538532 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 30091
I20260812 05:16:29.542349 30423 ts_tablet_manager.cc:1918] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:16:29.542502 30423 log.cc:1196] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/79fd928048834b0c868d3c96b75eec81
I20260812 05:16:29.542815 30423 ts_tablet_manager.cc:1939] T 79fd928048834b0c868d3c96b75eec81 P 2d9ef4a3af37400a9f293a03b4d0580b: Deleting consensus metadata
I20260812 05:16:29.543381 29869 catalog_manager.cc:4841] TS 2d9ef4a3af37400a9f293a03b4d0580b (127.18.166.195:43435): tablet 79fd928048834b0c868d3c96b75eec81 (table test-table [id=b8b752e69ee64ecbb7c65e7239533566]) successfully deleted
I20260812 05:16:29.545892 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 30233
I20260812 05:16:29.554545 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 29850
2026-08-12T05:16:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2219 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[30427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[30427](info): set up 2 sockets
Aug 12 05:16:29 dist-test-slave-srgv krb5kdc[30427](info): commencing operation
krb5kdc: starting...
W20260812 05:16:30.656610 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:16:30 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:16:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/rangeradmin_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/rangeradmin_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/rangerlookup_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/rangerlookup_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
I20260812 05:16:30.836336 19099 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskjJH1Yp/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/postgres -l logfile start
2026-08-12 05:16:33.519 UTC [30459] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 05:16:33.520 UTC [30459] LOG: listening on IPv4 address "127.18.166.212", port 41969
2026-08-12 05:16:33.524 UTC [30459] LOG: listening on Unix socket "/tmp/.s.PGSQL.41969"
2026-08-12 05:16:33.549 UTC [30462] LOG: database system was shut down at 2026-08-12 05:16:32 UTC
2026-08-12 05:16:33.553 UTC [30459] LOG: database system is ready to accept connections
I20260812 05:16:34.536531 19099 mini_postgres.cc:96] Postgres bound to 41969
2026-08-12 05:16:34.544 UTC [30470] FATAL: database "slave" does not exist
127.18.166.212:41969 - accepting connections
I20260812 05:16:34.544922 19099 mini_ranger.cc:161] Starting Ranger...
I20260812 05:16:34.561558 19099 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 05:16:35.042243 19099 mini_ranger.cc:87] Created ranger Postgres database
I20260812 05:16:35.042338 19099 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-admin
2026-08-12 05:16:35,366 [I] DB FLAVOR :POSTGRES
2026-08-12 05:16:35,366 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:16:35,367 [I] Checking connection..
2026-08-12 05:16:35,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 05:16:35,632 [I] Checking connection passed.
2026-08-12 05:16:35,633 [I] --------- Verifying version history table ---------
2026-08-12 05:16:35,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:16:35,893 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:16:35,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:16:36,171 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:16:36,171 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 05:16:36,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 05:16:36.424 UTC [30541] WARNING: there is no transaction in progress
2026-08-12 05:16:36,431 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 05:16:36,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:16:36,705 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 05:16:36,705 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:16:36,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 05:16:36,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 05:16:37,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-srgv.c.gcp-upstream.internal','N') ;"
2026-08-12 05:16:37,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:16:37,727 [I] Table x_portal_user does not exist in database ranger
2026-08-12 05:16:37,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:16:37,999 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 05:16:37,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:16:38,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:16:38,491 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 05:16:38,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 05:16:39.492 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.509 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.527 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.543 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.560 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.578 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.642 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.650 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.665 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.675 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.686 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.700 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.711 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.718 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.725 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:39.732 UTC [30661] WARNING: there is no transaction in progress
2026-08-12 05:16:40,400 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 05:16:40,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 05:16:40,682 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 05:16:40,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:16:40,927 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 05:16:40,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:16:41,166 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 05:16:41,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:16:41,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:16:41,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-srgv.c.gcp-upstream.internal';"
2026-08-12 05:16:41,920 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 05:16:41,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 05:16:42,161 [I] DB_PATCHES have already been applied
I20260812 05:16:42.167671 19099 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-admin:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskjJH1Yp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 05:16:42.167761 19099 mini_ranger.cc:193] Using host: 127.18.166.212
I20260812 05:16:42.171180 19099 mini_ranger.cc:227] Ranger admin URL: http://127.18.166.212:36367
Aug 12, 2026 5:16:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:16:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:16:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 5:16:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 5:16:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 5:16:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 5:16:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 05:16:43.311641 19099 mini_ranger.cc:160] Time spent starting Ranger: real 8.767s user 0.000s sys 0.010s
Aug 12, 2026 5:17:04 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 5:17:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 5:17:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 5:17:04 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 5:17:05 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 05:17:06.222137 19099 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/rangerkms_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/rangerkms_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.166.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.166.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 05:17:06 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511826, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 05:17:06.312496 19099 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 05:17:06.323464 19099 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 05:17:07.190343 19099 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 05:17:07.190436 19099 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms
I20260812 05:17:07.190456 19099 mini_ranger_kms.cc:227] Using postgres at 127.18.166.212:41969
2026-08-12 05:17:07,469 [I] DB FLAVOR :POSTGRES
2026-08-12 05:17:07,469 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:17:07,469 [I] Checking connection
2026-08-12 05:17:07,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 05:17:07,739 [I] connection success
2026-08-12 05:17:07,740 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 05:17:07,740 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 05:17:07,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 05:17:07,992 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 05:17:07,992 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:17:07,992 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 05:17:07,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:41969/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 05:17:08,268 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 05:17:08.751003 19099 mini_ranger_kms.cc:310] Created kms service
I20260812 05:17:08.936270 19099 mini_ranger_kms.cc:326] Created kudu user
I20260812 05:17:08.984483 19099 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 05:17:08 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511826, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:09.454824 19099 mini_ranger_kms.cc:384] Added ranger policy
I20260812 05:17:09.455135 19099 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms:/tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 05:17:09.455178 19099 mini_ranger_kms.cc:240] Using host: 127.18.166.212
I20260812 05:17:09.458981 19099 mini_ranger_kms.cc:276] Ranger KMS PID: 30856
I20260812 05:17:09.459120 19099 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.166.212:55369
Aug 12, 2026 5:17:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:17:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:17:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 5:17:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 5:17:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 5:17:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 05:17:11.850390 19099 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.538s user 0.000s sys 0.260s
I20260812 05:17:11.850519 19099 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 05:17:11.850617 19099 mini_ranger_kms.cc:401] 127.18.166.212:55369/kms/v1/keys
Aug 12 05:17:11 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786511831, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.166.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 05:17:12 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511831, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.166.212@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
Aug 12 05:17:12 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511831, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.166.212@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
Aug 12 05:17:15 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:17:15.358515 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:46245
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:46245
--ranger_config_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:55369/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:17:15.467905 30913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:17:15.468174 30913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:17:15.468216 30913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:17:15.469482 30913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:17:15.469543 30913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:17:15.469565 30913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:17:15.469606 30913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:17:15.469633 30913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:17:15.471294 30913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.166.212:55369/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:46245
--ranger_config_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:46245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:17:15.471572 30913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:17:15.471822 30913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:17:15.477807 30920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:15.477860 30919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:15.477807 30922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:17:15.478415 30913 server_base.cc:1034] running on GCE node
I20260812 05:17:15.478682 30913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:17:15.478987 30913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:17:15.480159 30913 hybrid_clock.cc:648] HybridClock initialized: now 1786511835480133 us; error 42 us; skew 500 ppm
Aug 12 05:17:15 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511835, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:17:15.482995 30913 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:17:15.483604 30913 webserver.cc:460] Webserver started at http://127.18.166.254:33145/ using document root <none> and password file <none>
I20260812 05:17:15.483846 30913 fs_manager.cc:359] Metadata directory not provided
I20260812 05:17:15.483908 30913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:17:15.484090 30913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:17:15 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511835, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:15.540531 30913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "5ee99fc3cd894423a170a85147adf230"
format_stamp: "Formatted at 2026-08-12 05:17:15 on dist-test-slave-srgv"
server_key: "558564aa1cd76774cbae999d93cad32c"
server_key_iv: "c130ba0ef933ae5f7f6db108ace5b3d8"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:15.540961 30913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "5ee99fc3cd894423a170a85147adf230"
format_stamp: "Formatted at 2026-08-12 05:17:15 on dist-test-slave-srgv"
server_key: "558564aa1cd76774cbae999d93cad32c"
server_key_iv: "c130ba0ef933ae5f7f6db108ace5b3d8"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:15.542615 30913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:17:15.558077 30929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:15.558333 30913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:17:15.558429 30913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "5ee99fc3cd894423a170a85147adf230"
format_stamp: "Formatted at 2026-08-12 05:17:15 on dist-test-slave-srgv"
server_key: "558564aa1cd76774cbae999d93cad32c"
server_key_iv: "c130ba0ef933ae5f7f6db108ace5b3d8"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:15.558522 30913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:17:15.566795 30913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:17:15.568853 30913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:17:15.569051 30913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:17:15.574697 30913 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:46245
I20260812 05:17:15.574721 30987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:46245 every 8 connection(s)
I20260812 05:17:15.575132 30913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:17:15.575953 30988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:17:15.576226 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 30913
I20260812 05:17:15.576352 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:17:15.576558 19099 external_mini_cluster.cc:1428] Setting key 7faf4e8036fd4d5ee184b3b7b9e0f906
I20260812 05:17:15.579104 30988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230: Bootstrap starting.
I20260812 05:17:15.579914 30988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230: Neither blocks nor log segments found. Creating new log.
I20260812 05:17:15.580334 30988 log.cc:824] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230: Log is configured to *not* fsync() on all Append() calls
I20260812 05:17:15.581866 30988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230: No bootstrap required, opened a new log
Aug 12 05:17:15 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:17:15.584555 30988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee99fc3cd894423a170a85147adf230" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 46245 } }
I20260812 05:17:15.584764 30988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:17:15.584805 30988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ee99fc3cd894423a170a85147adf230, State: Initialized, Role: FOLLOWER
I20260812 05:17:15.584976 30988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee99fc3cd894423a170a85147adf230" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 46245 } }
I20260812 05:17:15.585081 30988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:17:15.585146 30988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:17:15.585211 30988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:17:15.586066 30988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee99fc3cd894423a170a85147adf230" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 46245 } }
I20260812 05:17:15.586182 30988 leader_election.cc:302] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ee99fc3cd894423a170a85147adf230; no voters:
I20260812 05:17:15.586413 30988 leader_election.cc:288] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:17:15.586570 30993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:17:15.586751 30988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:17:15.586776 30993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [term 1 LEADER]: Becoming Leader. State: Replica: 5ee99fc3cd894423a170a85147adf230, State: Running, Role: LEADER
I20260812 05:17:15.586954 30993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee99fc3cd894423a170a85147adf230" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 46245 } }
I20260812 05:17:15.587329 30993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ee99fc3cd894423a170a85147adf230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee99fc3cd894423a170a85147adf230" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 46245 } } }
I20260812 05:17:15.587385 30995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ee99fc3cd894423a170a85147adf230. Latest consensus state: current_term: 1 leader_uuid: "5ee99fc3cd894423a170a85147adf230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee99fc3cd894423a170a85147adf230" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 46245 } } }
I20260812 05:17:15.587462 30993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [sys.catalog]: This master's current role is: LEADER
I20260812 05:17:15.587479 30995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230 [sys.catalog]: This master's current role is: LEADER
I20260812 05:17:15.587857 30997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:17:15.588423 30997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:17:15.589048 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:15.577128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.18.166.254:46245)
0812 05:17:15.577322 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:15.577332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:15.577435 (+ 103us) server_negotiation.cc:407] Connection header received
0812 05:17:15.578061 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:15.578067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:15.578126 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:15.578299 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:17:15.578962 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:15.579904 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:15.580803 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:15.581475 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:15.581602 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:15.584606 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:17:15.584610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:17:15.584611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:17:15.584624 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:17:15.586749 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:15.587218 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:15.587221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:15.587221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:15.587308 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:15.587515 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:15.587516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:15.587516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:15.587850 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:17:15.588001 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:15.588601 (+ 600us) server_negotiation.cc:299] Negotiation successful
0812 05:17:15.588744 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260812 05:17:15.589084 30988 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 05:17:15.593873 30997 catalog_manager.cc:1347] Generated new cluster ID: e6a8f3047cac4f3aa2e43626282435b6
I20260812 05:17:15.594076 30997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:17:15.622649 30997 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:17:15.624336 30997 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:17:15.629283 30997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230: Generated new TSK 0
I20260812 05:17:15.629601 30997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:17:16.437193 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:55369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:46245
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:17:16.612469 31035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:17:16.612972 31035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:17:16.613042 31035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:17:16.615279 31035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:17:16.615351 31035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:17:16.615396 31035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:17:16.618352 31035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.166.212:55369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:46245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:17:16.618701 31035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:17:16.619076 31035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:17:16.625612 31041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:16.625622 31040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:16.625792 31043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:17:16.626024 31035 server_base.cc:1034] running on GCE node
I20260812 05:17:16.626224 31035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:17:16.626513 31035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:17:16.627708 31035 hybrid_clock.cc:648] HybridClock initialized: now 1786511836627695 us; error 47 us; skew 500 ppm
Aug 12 05:17:16 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:17:16.631753 31035 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:17:16.632191 31035 webserver.cc:460] Webserver started at http://127.18.166.193:33833/ using document root <none> and password file <none>
I20260812 05:17:16.632434 31035 fs_manager.cc:359] Metadata directory not provided
I20260812 05:17:16.632505 31035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:17:16.632668 31035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:17:16 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:16.650307 31035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "2eacfd43f086469a8813582434212ca7"
format_stamp: "Formatted at 2026-08-12 05:17:16 on dist-test-slave-srgv"
server_key: "003287325c8dd292958fdb34a697801e"
server_key_iv: "8b17722f1254f78d64fea9b9908d7873"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:16.651085 31035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "2eacfd43f086469a8813582434212ca7"
format_stamp: "Formatted at 2026-08-12 05:17:16 on dist-test-slave-srgv"
server_key: "003287325c8dd292958fdb34a697801e"
server_key_iv: "8b17722f1254f78d64fea9b9908d7873"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:16.654045 31035 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 05:17:16.668356 31051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:16.668609 31035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:17:16.668705 31035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "2eacfd43f086469a8813582434212ca7"
format_stamp: "Formatted at 2026-08-12 05:17:16 on dist-test-slave-srgv"
server_key: "003287325c8dd292958fdb34a697801e"
server_key_iv: "8b17722f1254f78d64fea9b9908d7873"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:16.668814 31035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:17:16.684892 31035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:17:16.687691 31035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:17:16.687958 31035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:17:16.688359 31035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:17:16.688881 31035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:17:16.688953 31035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:16.688992 31035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:17:16.689066 31035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:16.705647 31035 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:38247
I20260812 05:17:16.705806 31174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:38247 every 8 connection(s)
I20260812 05:17:16.706234 31035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:17:16.709282 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 31035
I20260812 05:17:16.709441 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:17:16.709656 19099 external_mini_cluster.cc:1428] Setting key 2a18ad1876a7f8b8bfa5f11e8cbdaa34
Aug 12 05:17:16 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:17:16.724512 31178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:16.707169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:46245 (local address 127.18.166.193:54601)
0812 05:17:16.707505 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:16.707539 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:16.708207 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:16.708479 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:16.708482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:16.708811 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:17:16.709192 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:16.709200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:16.712249 (+ 3049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:16.712251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:16.713846 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:16.713850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:16.714414 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:16.714416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:16.714646 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:16.716696 (+ 2050us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:17:16.716708 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:17:16.719215 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:17:16.721919 (+ 2704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:17:16.721934 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:17:16.721937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:17:16.722441 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:17:16.722834 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:17:16.722835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:17:16.722835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:17:16.723034 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:17:16.723615 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:17:16.723624 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:17:16.723851 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 05:17:16.723984 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 05:17:16.724205 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
I20260812 05:17:16.725294 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:16.707221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:54601 (local address 127.18.166.254:46245)
0812 05:17:16.707435 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:16.707437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:16.707647 (+ 210us) server_negotiation.cc:407] Connection header received
0812 05:17:16.708299 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:16.708301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:16.708316 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:16.708408 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:17:16.709347 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:16.710296 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:16.713964 (+ 3668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:16.714299 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:16.714387 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:16.719407 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:17:16.719410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:17:16.719410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:17:16.719420 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:17:16.721796 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:16.722556 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:16.722559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:16.722559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:16.722669 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:16.723326 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:16.723328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:16.723328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:16.723523 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:17:16.723611 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:16.725089 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0812 05:17:16.725192 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
I20260812 05:17:16.725884 31175 heartbeater.cc:347] Connected to a master server at 127.18.166.254:46245
I20260812 05:17:16.726011 31175 heartbeater.cc:464] Registering TS with master...
I20260812 05:17:16.726222 31175 heartbeater.cc:511] Master 127.18.166.254:46245 requested a full tablet report, sending...
I20260812 05:17:16.726826 30949 ts_manager.cc:194] Registered new tserver with Master: 2eacfd43f086469a8813582434212ca7 (127.18.166.193:38247)
I20260812 05:17:16.727782 30949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:54601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:17:16.744721 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:55369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:46245
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:17:16.891317 31183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:17:16.891618 31183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:17:16.891652 31183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:17:16.893671 31183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:17:16.893834 31183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:17:16.893923 31183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:17:16.896817 31183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.166.212:55369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:46245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:17:16.897238 31183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:17:16.897643 31183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:17:16.905961 31183 server_base.cc:1034] running on GCE node
W20260812 05:17:16.905853 31194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:16.906153 31192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:16.906673 31191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:17:16.907050 31183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:17:16.907574 31183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:17:16.908761 31183 hybrid_clock.cc:648] HybridClock initialized: now 1786511836908725 us; error 52 us; skew 500 ppm
Aug 12 05:17:16 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:17:16.913789 31183 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:17:16.914510 31183 webserver.cc:460] Webserver started at http://127.18.166.194:45859/ using document root <none> and password file <none>
I20260812 05:17:16.914819 31183 fs_manager.cc:359] Metadata directory not provided
I20260812 05:17:16.914904 31183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:17:16.915144 31183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:17:16 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:16.941314 31183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "c00ab14d1bcb4c18b79bc3071897f6da"
format_stamp: "Formatted at 2026-08-12 05:17:16 on dist-test-slave-srgv"
server_key: "741ee2951f102b8e61e5481e737b8a43"
server_key_iv: "ee3a2a81104cceb32c65b8eaaf2c8e9f"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:16.941982 31183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "c00ab14d1bcb4c18b79bc3071897f6da"
format_stamp: "Formatted at 2026-08-12 05:17:16 on dist-test-slave-srgv"
server_key: "741ee2951f102b8e61e5481e737b8a43"
server_key_iv: "ee3a2a81104cceb32c65b8eaaf2c8e9f"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:16.944119 31183 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 05:17:16.959004 31202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:16.959292 31183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 05:17:16.959401 31183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "c00ab14d1bcb4c18b79bc3071897f6da"
format_stamp: "Formatted at 2026-08-12 05:17:16 on dist-test-slave-srgv"
server_key: "741ee2951f102b8e61e5481e737b8a43"
server_key_iv: "ee3a2a81104cceb32c65b8eaaf2c8e9f"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:16.959535 31183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 12 05:17:16 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:17:16.975971 31183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:17:16.979013 31183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:17:16.979353 31183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:17:16.979789 31183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:17:16.980309 31183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:17:16.980409 31183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:16.980490 31183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:17:16.980556 31183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:16.999704 31183 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41393
I20260812 05:17:17.000447 31183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:17:17.001245 31323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41393 every 8 connection(s)
I20260812 05:17:17.003626 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 31183
I20260812 05:17:17.003742 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:17:17.004045 19099 external_mini_cluster.cc:1428] Setting key 5e34c8bf353a01a44bcf62345951a069
Aug 12 05:17:17 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:17:17.015586 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.002834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:51287 (local address 127.18.166.254:46245)
0812 05:17:17.002887 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.002890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.003179 (+ 289us) server_negotiation.cc:407] Connection header received
0812 05:17:17.004445 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.004446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.004462 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.004493 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:17:17.005659 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.006432 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.008276 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.008571 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.008615 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.011908 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:17:17.011910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:17:17.011911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:17:17.011920 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:17:17.013827 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:17.014398 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:17.014399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:17.014400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:17.014466 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:17.014772 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:17.014773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:17.014773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:17.014916 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:17:17.014963 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.015383 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.015483 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:17:17.016505 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.002451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:46245 (local address 127.18.166.194:51287)
0812 05:17:17.003017 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.003066 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.004332 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.004675 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.004680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.005102 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:17:17.005542 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.005548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.006572 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.006573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.008150 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.008153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.008713 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.008714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.008912 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.009677 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:17:17.009688 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:17:17.011771 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:17:17.013938 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:17:17.013948 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:17:17.013951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:17:17.014297 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:17:17.014543 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:17:17.014544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:17:17.014544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:17:17.014692 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:17:17.015015 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:17:17.015021 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:17:17.015210 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.016008 (+ 798us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.016209 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":172,"threads_started":1}
I20260812 05:17:17.017275 31324 heartbeater.cc:347] Connected to a master server at 127.18.166.254:46245
I20260812 05:17:17.017376 31324 heartbeater.cc:464] Registering TS with master...
I20260812 05:17:17.017597 31324 heartbeater.cc:511] Master 127.18.166.254:46245 requested a full tablet report, sending...
I20260812 05:17:17.018174 30949 ts_manager.cc:194] Registered new tserver with Master: c00ab14d1bcb4c18b79bc3071897f6da (127.18.166.194:41393)
I20260812 05:17:17.018672 30949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:51287
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:17:17.053886 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:55369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:46245
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:36945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:17:17.213521 31329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:17:17.214068 31329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:17:17.214210 31329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:17:17.216198 31329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:17:17.216336 31329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:17:17.216427 31329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:17:17.219158 31329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:36945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.166.212:55369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:46245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:17:17.219779 31329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:17:17.220160 31329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:17:17.229140 31336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:17:17.229591 31340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:17:17.230793 31329 server_base.cc:1034] running on GCE node
W20260812 05:17:17.231063 31337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:17:17.231385 31329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:17:17.231711 31329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:17:17.232966 31329 hybrid_clock.cc:648] HybridClock initialized: now 1786511837232934 us; error 47 us; skew 500 ppm
Aug 12 05:17:17 dist-test-slave-srgv krb5kdc[30427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511837, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:17:17.237274 31329 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:17:17.237906 31329 webserver.cc:460] Webserver started at http://127.18.166.195:36891/ using document root <none> and password file <none>
I20260812 05:17:17.238262 31329 fs_manager.cc:359] Metadata directory not provided
I20260812 05:17:17.238381 31329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:17:17.238582 31329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:17:17 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511837, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:17.256690 31329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "cf1044c79359427db90dedfdea5dacc8"
format_stamp: "Formatted at 2026-08-12 05:17:17 on dist-test-slave-srgv"
server_key: "c71e3c993b55c3bcb808bcb640a29d79"
server_key_iv: "5c0a7271900885700c883b69f8fc52bd"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:17.257238 31329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "cf1044c79359427db90dedfdea5dacc8"
format_stamp: "Formatted at 2026-08-12 05:17:17 on dist-test-slave-srgv"
server_key: "c71e3c993b55c3bcb808bcb640a29d79"
server_key_iv: "5c0a7271900885700c883b69f8fc52bd"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:17.262981 31329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260812 05:17:17.279489 31347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:17.279732 31329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:17:17.279855 31329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "cf1044c79359427db90dedfdea5dacc8"
format_stamp: "Formatted at 2026-08-12 05:17:17 on dist-test-slave-srgv"
server_key: "c71e3c993b55c3bcb808bcb640a29d79"
server_key_iv: "5c0a7271900885700c883b69f8fc52bd"
server_key_version: "kuduclusterkey@0"
I20260812 05:17:17.279961 31329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:17:17.297070 31329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:17:17.300253 31329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:17:17.300503 31329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:17:17.301157 31329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:17:17.301911 31329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:17:17.301985 31329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:17.302021 31329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:17:17.302109 31329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:17:17.318416 31329 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:45547
I20260812 05:17:17.318953 31329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:17:17.319928 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 31329
I20260812 05:17:17.320019 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:17:17.320226 19099 external_mini_cluster.cc:1428] Setting key ed3416b3117fe9969222969c6a88b753
I20260812 05:17:17.322813 31463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:45547 every 8 connection(s)
Aug 12 05:17:17 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511837, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:17:17.336108 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.322528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:41289 (local address 127.18.166.254:46245)
0812 05:17:17.322577 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.322580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.324123 (+ 1543us) server_negotiation.cc:407] Connection header received
0812 05:17:17.325115 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.325117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.325129 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.325163 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:17:17.326289 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.326984 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.328416 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.328690 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.328734 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.332410 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:17:17.332412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:17:17.332412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:17:17.332422 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:17:17.334397 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:17.334954 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:17.334955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:17.334956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:17.335019 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:17.335321 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:17.335322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:17.335323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:17.335466 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:17:17.335511 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.335913 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.336006 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:17:17.336894 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.322140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:46245 (local address 127.18.166.195:41289)
0812 05:17:17.323961 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.324011 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.324987 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.325369 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.325374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.325776 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:17:17.326154 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.326162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.327104 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.327105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.328312 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.328315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.328828 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.328828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.328961 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.329740 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:17:17.329750 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:17:17.332296 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:17:17.334507 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:17:17.334515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:17:17.334518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:17:17.334858 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:17:17.335096 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:17:17.335097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:17:17.335098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:17:17.335241 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:17:17.335558 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:17:17.335564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:17:17.335752 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.336388 (+ 636us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.336578 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1769,"thread_start_us":192,"threads_started":1}
I20260812 05:17:17.337625 31464 heartbeater.cc:347] Connected to a master server at 127.18.166.254:46245
I20260812 05:17:17.337725 31464 heartbeater.cc:464] Registering TS with master...
I20260812 05:17:17.337929 31464 heartbeater.cc:511] Master 127.18.166.254:46245 requested a full tablet report, sending...
I20260812 05:17:17.338356 30949 ts_manager.cc:194] Registered new tserver with Master: cf1044c79359427db90dedfdea5dacc8 (127.18.166.195:45547)
I20260812 05:17:17.339008 30949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:41289
I20260812 05:17:17.343243 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:17:17.357247 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.345218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.18.166.254:46245)
0812 05:17:17.345269 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.345271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.345357 (+ 86us) server_negotiation.cc:407] Connection header received
0812 05:17:17.345525 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.345526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.345548 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.345591 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:17:17.348211 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.349045 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.350353 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.350680 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.350757 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.353162 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:17:17.353163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:17:17.353164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:17:17.353172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:17:17.355462 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:17.355990 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:17.355991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:17.355992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:17.356069 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:17:17.356371 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:17:17.356372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:17:17.356372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:17:17.356552 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:17:17.356645 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.356965 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.357115 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:17:17.359977 30949 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:17:17.363832 30949 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:17:17.381464 31482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.368589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45547 (local address 127.0.0.1:44266)
0812 05:17:17.374279 (+ 5690us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.374300 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.374368 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.374897 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.374901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.374904 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:17:17.374990 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.374992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.377330 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.377331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.379104 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.379107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.380977 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.380979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.381115 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.381141 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.381235 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.381313 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5656,"thread_start_us":111,"threads_started":1}
I20260812 05:17:17.382089 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.369146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.18.166.195:45547)
0812 05:17:17.369480 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.369487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.374534 (+ 5047us) server_negotiation.cc:407] Connection header received
0812 05:17:17.374634 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.374637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.374657 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.374743 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:17:17.375378 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.377207 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.379303 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.380735 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.380915 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.380989 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.381854 (+ 865us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.381932 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":158,"threads_started":1}
I20260812 05:17:17.383234 31397 tablet_service.cc:1467] Processing CreateTablet for tablet 10b3f1bb077e42558c8006ddf402026a (DEFAULT_TABLE table=test-table [id=bd054d07354045dd994b2283648899fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:17:17.383733 31397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10b3f1bb077e42558c8006ddf402026a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:17:17.389272 31486 tablet_bootstrap.cc:492] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Bootstrap starting.
I20260812 05:17:17.391063 31486 tablet_bootstrap.cc:654] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Neither blocks nor log segments found. Creating new log.
I20260812 05:17:17.391829 31486 log.cc:824] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Log is configured to *not* fsync() on all Append() calls
I20260812 05:17:17.392005 31480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.368859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41393 (local address 127.0.0.1:36870)
0812 05:17:17.375151 (+ 6292us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.375174 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.375219 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.381453 (+ 6234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.381454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.381456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:17:17.381503 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.381505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.384941 (+ 3436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.384942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.386547 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.386551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.391749 (+ 5198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.391751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.391823 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.391830 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.391867 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.391920 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6259,"thread_start_us":121,"threads_started":1}
I20260812 05:17:17.392874 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.369000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.18.166.194:41393)
0812 05:17:17.381218 (+ 12218us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.381224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.381235 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:17:17.381295 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.381297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.381315 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.381587 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:17:17.381667 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.382892 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.386834 (+ 3942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.388550 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.392565 (+ 4015us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.392659 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.392697 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.392750 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12147,"thread_start_us":176,"threads_started":1}
I20260812 05:17:17.393000 31486 tablet_bootstrap.cc:492] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: No bootstrap required, opened a new log
I20260812 05:17:17.393069 31486 ts_tablet_manager.cc:1397] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 05:17:17.393860 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.368292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:38247 (local address 127.0.0.1:38052)
0812 05:17:17.373599 (+ 5307us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.373677 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.373776 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.383254 (+ 9478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.383255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.383257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:17:17.383325 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.383327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.388665 (+ 5338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.388666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.389960 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.389964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.393521 (+ 3557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.393523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.393644 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.393649 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.393695 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.393780 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5248,"thread_start_us":140,"threads_started":1}
I20260812 05:17:17.394411 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.368435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.18.166.193:38247)
0812 05:17:17.383042 (+ 14607us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.383047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.383057 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:17:17.383122 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.383125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.383143 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.383413 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:17:17.383483 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.388106 (+ 4623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.390541 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.391566 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.394113 (+ 2547us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.394216 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.394256 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.394334 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14564,"thread_start_us":101,"threads_started":1}
I20260812 05:17:17.394274 31255 tablet_service.cc:1467] Processing CreateTablet for tablet 10b3f1bb077e42558c8006ddf402026a (DEFAULT_TABLE table=test-table [id=bd054d07354045dd994b2283648899fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:17:17.394896 31255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10b3f1bb077e42558c8006ddf402026a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:17:17.395149 31104 tablet_service.cc:1467] Processing CreateTablet for tablet 10b3f1bb077e42558c8006ddf402026a (DEFAULT_TABLE table=test-table [id=bd054d07354045dd994b2283648899fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:17:17.395610 31104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10b3f1bb077e42558c8006ddf402026a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:17:17.396814 31486 raft_consensus.cc:348] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.397022 31486 raft_consensus.cc:374] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:17:17.397071 31486 raft_consensus.cc:729] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1044c79359427db90dedfdea5dacc8, State: Initialized, Role: FOLLOWER
I20260812 05:17:17.397810 31486 consensus_queue.cc:260] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.398406 31486 ts_tablet_manager.cc:1428] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 05:17:17.398597 31464 heartbeater.cc:503] Master 127.18.166.254:46245 was elected leader, sending a full tablet report...
I20260812 05:17:17.400540 31489 tablet_bootstrap.cc:492] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: Bootstrap starting.
I20260812 05:17:17.402220 31489 tablet_bootstrap.cc:654] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: Neither blocks nor log segments found. Creating new log.
I20260812 05:17:17.402839 31489 log.cc:824] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: Log is configured to *not* fsync() on all Append() calls
I20260812 05:17:17.403311 31490 tablet_bootstrap.cc:492] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Bootstrap starting.
I20260812 05:17:17.403987 31489 tablet_bootstrap.cc:492] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: No bootstrap required, opened a new log
I20260812 05:17:17.404165 31489 ts_tablet_manager.cc:1397] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 05:17:17.405187 31490 tablet_bootstrap.cc:654] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Neither blocks nor log segments found. Creating new log.
I20260812 05:17:17.406114 31490 log.cc:824] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Log is configured to *not* fsync() on all Append() calls
I20260812 05:17:17.407382 31490 tablet_bootstrap.cc:492] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: No bootstrap required, opened a new log
I20260812 05:17:17.407500 31490 ts_tablet_manager.cc:1397] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260812 05:17:17.410832 31489 raft_consensus.cc:348] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.411136 31489 raft_consensus.cc:374] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:17:17.411193 31489 raft_consensus.cc:729] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eacfd43f086469a8813582434212ca7, State: Initialized, Role: FOLLOWER
I20260812 05:17:17.411412 31489 consensus_queue.cc:260] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.411491 31490 raft_consensus.cc:348] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.411684 31490 raft_consensus.cc:374] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:17:17.411729 31490 raft_consensus.cc:729] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c00ab14d1bcb4c18b79bc3071897f6da, State: Initialized, Role: FOLLOWER
I20260812 05:17:17.411849 31489 ts_tablet_manager.cc:1428] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260812 05:17:17.411856 31490 consensus_queue.cc:260] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.411990 31175 heartbeater.cc:503] Master 127.18.166.254:46245 was elected leader, sending a full tablet report...
I20260812 05:17:17.412195 31490 ts_tablet_manager.cc:1428] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 05:17:17.412360 31324 heartbeater.cc:503] Master 127.18.166.254:46245 was elected leader, sending a full tablet report...
W20260812 05:17:17.457588 31176 tablet.cc:2369] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:17:17.502158 31325 tablet.cc:2369] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:17:17.573874 31465 tablet.cc:2369] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 05:17:17 dist-test-slave-srgv krb5kdc[30427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511836, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:17.620965 31494 raft_consensus.cc:482] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:17:17.621129 31494 raft_consensus.cc:504] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.621610 31494 leader_election.cc:288] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2eacfd43f086469a8813582434212ca7 (127.18.166.193:38247), cf1044c79359427db90dedfdea5dacc8 (127.18.166.195:45547)
I20260812 05:17:17.626709 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.622351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:55101 (local address 127.18.166.193:38247)
0812 05:17:17.622396 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.622398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.622539 (+ 141us) server_negotiation.cc:407] Connection header received
0812 05:17:17.622668 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.622670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.622681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.622712 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:17:17.623002 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.623786 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.625361 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.626227 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.626285 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.626295 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.626574 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.626631 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:17:17.627244 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.621924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:38247 (local address 127.18.166.194:55101)
0812 05:17:17.622466 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.622478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.622568 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.622781 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.622782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.622785 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:17:17.622921 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.622923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.623936 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.623937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.625249 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.625252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.626367 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.626367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.626484 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.626489 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.627059 (+ 570us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.627112 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":69,"threads_started":1}
I20260812 05:17:17.628227 31125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b3f1bb077e42558c8006ddf402026a" candidate_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eacfd43f086469a8813582434212ca7" is_pre_election: true
I20260812 05:17:17.628625 31125 raft_consensus.cc:2454] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c00ab14d1bcb4c18b79bc3071897f6da in term 0.
I20260812 05:17:17.628988 31204 leader_election.cc:302] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eacfd43f086469a8813582434212ca7, c00ab14d1bcb4c18b79bc3071897f6da; no voters:
I20260812 05:17:17.629155 31494 raft_consensus.cc:2781] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:17:17.629204 31494 raft_consensus.cc:482] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:17:17.629225 31494 raft_consensus.cc:3037] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:17:17.630753 31494 raft_consensus.cc:504] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.630904 31494 leader_election.cc:288] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [CANDIDATE]: Term 1 election: Requested vote from peers 2eacfd43f086469a8813582434212ca7 (127.18.166.193:38247), cf1044c79359427db90dedfdea5dacc8 (127.18.166.195:45547)
I20260812 05:17:17.631251 31125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b3f1bb077e42558c8006ddf402026a" candidate_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eacfd43f086469a8813582434212ca7"
I20260812 05:17:17.631371 31125 raft_consensus.cc:3037] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:17:17.631824 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.622199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:35815 (local address 127.18.166.195:45547)
0812 05:17:17.622255 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.622257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.627423 (+ 5166us) server_negotiation.cc:407] Connection header received
0812 05:17:17.627582 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.627584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.627596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.627666 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:17:17.627955 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.628750 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.630730 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.631348 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.631403 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.631412 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.631708 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.631754 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:17:17.632218 31496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.622099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45547 (local address 127.18.166.194:35815)
0812 05:17:17.627337 (+ 5238us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.627353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:17:17.627481 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:17:17.627744 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:17:17.627746 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:17:17.627749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:17:17.627836 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.627838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.628954 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.628955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.630614 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:17:17.630617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.631479 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.631480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:17:17.631609 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.631613 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:17:17.631941 (+ 328us) client_negotiation.cc:240] Negotiation successful
0812 05:17:17.631994 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5217,"thread_start_us":71,"threads_started":1}
I20260812 05:17:17.632700 31125 raft_consensus.cc:2454] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c00ab14d1bcb4c18b79bc3071897f6da in term 1.
I20260812 05:17:17.632894 31204 leader_election.cc:302] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eacfd43f086469a8813582434212ca7, c00ab14d1bcb4c18b79bc3071897f6da; no voters:
I20260812 05:17:17.632874 31403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b3f1bb077e42558c8006ddf402026a" candidate_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf1044c79359427db90dedfdea5dacc8" is_pre_election: true
I20260812 05:17:17.633028 31403 raft_consensus.cc:2454] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c00ab14d1bcb4c18b79bc3071897f6da in term 0.
I20260812 05:17:17.633263 31402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b3f1bb077e42558c8006ddf402026a" candidate_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf1044c79359427db90dedfdea5dacc8"
I20260812 05:17:17.633447 31402 raft_consensus.cc:3037] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:17:17.634537 31402 raft_consensus.cc:2454] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c00ab14d1bcb4c18b79bc3071897f6da in term 1.
I20260812 05:17:17.634750 31494 raft_consensus.cc:2781] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:17:17.634929 31494 raft_consensus.cc:686] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 1 LEADER]: Becoming Leader. State: Replica: c00ab14d1bcb4c18b79bc3071897f6da, State: Running, Role: LEADER
I20260812 05:17:17.635038 31494 consensus_queue.cc:237] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } }
I20260812 05:17:17.635871 30947 catalog_manager.cc:5495] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da reported cstate change: term changed from 0 to 1, leader changed from <none> to c00ab14d1bcb4c18b79bc3071897f6da (127.18.166.194). New cstate: current_term: 1 leader_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c00ab14d1bcb4c18b79bc3071897f6da" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41393 } health_report { overall_health: HEALTHY } } }
I20260812 05:17:17.674515 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.670402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36876 (local address 127.18.166.194:41393)
0812 05:17:17.670452 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:17:17.670454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:17:17.670596 (+ 142us) server_negotiation.cc:407] Connection header received
0812 05:17:17.670739 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:17:17.670740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:17:17.670768 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:17:17.670874 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:17:17.671080 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.671852 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.673015 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:17:17.673437 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:17:17.673506 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:17:17.673800 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:17:17.673938 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:17:17.674041 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 05:17:17.674321 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 05:17:17.674344 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:17:17.678850 31402 raft_consensus.cc:1261] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 1 FOLLOWER]: Refusing update from remote peer c00ab14d1bcb4c18b79bc3071897f6da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:17:17.679361 31494 consensus_queue.cc:1035] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf1044c79359427db90dedfdea5dacc8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:17:17.682363 31125 raft_consensus.cc:1261] T 10b3f1bb077e42558c8006ddf402026a P 2eacfd43f086469a8813582434212ca7 [term 1 FOLLOWER]: Refusing update from remote peer c00ab14d1bcb4c18b79bc3071897f6da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:17:17.682796 31494 consensus_queue.cc:1035] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eacfd43f086469a8813582434212ca7" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 38247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:17:17.686765 31504 mvcc.cc:205] Tried to move back new op lower bound from 7317552487128457216 to 7317552486953578496. Current Snapshot: MvccSnapshot[applied={T|T < 7317552487128457216}]
I20260812 05:17:17.687294 31501 mvcc.cc:205] Tried to move back new op lower bound from 7317552487128457216 to 7317552486953578496. Current Snapshot: MvccSnapshot[applied={T|T < 7317552487128457216}]
I20260812 05:17:17.693022 30947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:17:17.693176 30947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:17:17.694281 30947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ee99fc3cd894423a170a85147adf230: Sending DeleteTablet for 3 replicas of tablet 10b3f1bb077e42558c8006ddf402026a
I20260812 05:17:17.694967 31104 tablet_service.cc:1514] Processing DeleteTablet for tablet 10b3f1bb077e42558c8006ddf402026a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:17:17 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:38052
I20260812 05:17:17.695463 31397 tablet_service.cc:1514] Processing DeleteTablet for tablet 10b3f1bb077e42558c8006ddf402026a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:17:17 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44266
I20260812 05:17:17.695837 31255 tablet_service.cc:1514] Processing DeleteTablet for tablet 10b3f1bb077e42558c8006ddf402026a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:17:17 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:36870
I20260812 05:17:17.697116 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 31035
I20260812 05:17:17.701608 31513 tablet_replica.cc:330] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: stopping tablet replica
I20260812 05:17:17.701886 31514 tablet_replica.cc:330] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: stopping tablet replica
I20260812 05:17:17.707077 31514 raft_consensus.cc:2229] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:17:17.707379 31513 raft_consensus.cc:2229] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:17:17.707646 31514 raft_consensus.cc:2258] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:17:17.707861 31513 raft_consensus.cc:2258] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:17:17.708699 31513 ts_tablet_manager.cc:1905] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:17:17.709172 31514 ts_tablet_manager.cc:1905] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:17:17.711256 31514 ts_tablet_manager.cc:1918] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:17:17.713256 31513 ts_tablet_manager.cc:1918] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:17:17.713454 31513 log.cc:1196] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/10b3f1bb077e42558c8006ddf402026a
I20260812 05:17:17.714215 31513 ts_tablet_manager.cc:1939] T 10b3f1bb077e42558c8006ddf402026a P cf1044c79359427db90dedfdea5dacc8: Deleting consensus metadata
I20260812 05:17:17.714389 31514 log.cc:1196] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/10b3f1bb077e42558c8006ddf402026a
I20260812 05:17:17.714754 31514 ts_tablet_manager.cc:1939] T 10b3f1bb077e42558c8006ddf402026a P c00ab14d1bcb4c18b79bc3071897f6da: Deleting consensus metadata
I20260812 05:17:17.715379 30930 catalog_manager.cc:4841] TS c00ab14d1bcb4c18b79bc3071897f6da (127.18.166.194:41393): tablet 10b3f1bb077e42558c8006ddf402026a (table test-table [id=bd054d07354045dd994b2283648899fa]) successfully deleted
I20260812 05:17:17.715992 30931 catalog_manager.cc:4841] TS cf1044c79359427db90dedfdea5dacc8 (127.18.166.195:45547): tablet 10b3f1bb077e42558c8006ddf402026a (table test-table [id=bd054d07354045dd994b2283648899fa]) successfully deleted
W20260812 05:17:17.716356 30931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:38247 (error 108)
I20260812 05:17:17.717437 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:17:17.717283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:38247 (local address 127.0.0.1:38066)
0812 05:17:17.717334 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:17:17.717375 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 05:17:17.717485 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 31183
W20260812 05:17:17.719908 30931 catalog_manager.cc:4584] TS 2eacfd43f086469a8813582434212ca7 (127.18.166.193:38247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10b3f1bb077e42558c8006ddf402026a: Network error: Client connection negotiation failed: client connection to 127.18.166.193:38247: connect: Connection refused (error 111)
I20260812 05:17:17.728040 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 31329
I20260812 05:17:17.734551 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 30913
I20260812 05:17:17.743382 19099 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 05:17:18.088294 19099 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 05:17:18.088451 19099 mini_ranger.cc:66] Stopping Ranger...
I20260812 05:17:18.434109 19099 mini_ranger.cc:68] Stopped Ranger
2026-08-12 05:17:18.434 UTC [30459] LOG: received smart shutdown request
2026-08-12 05:17:18.438 UTC [30459] LOG: background worker "logical replication launcher" (PID 30468) exited with exit code 1
2026-08-12 05:17:18.438 UTC [30463] LOG: shutting down
2026-08-12 05:17:18.614 UTC [30459] LOG: database system is shut down
2026-08-12T05:17:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49170 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:17:18 dist-test-slave-srgv krb5kdc[31525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:17:18 dist-test-slave-srgv krb5kdc[31525](info): set up 2 sockets
Aug 12 05:17:18 dist-test-slave-srgv krb5kdc[31525](info): commencing operation
krb5kdc: starting...
W20260812 05:17:19.795120 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:17:19 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:17:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:17:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/rangeradmin_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/rangeradmin_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/rangerlookup_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/rangerlookup_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
I20260812 05:17:19.980182 19099 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskjJH1Yp/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/postgres -l logfile start
2026-08-12 05:17:22.550 UTC [31559] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 05:17:22.550 UTC [31559] LOG: listening on IPv4 address "127.18.166.212", port 38315
2026-08-12 05:17:22.554 UTC [31559] LOG: listening on Unix socket "/tmp/.s.PGSQL.38315"
2026-08-12 05:17:22.576 UTC [31562] LOG: database system was shut down at 2026-08-12 05:17:21 UTC
2026-08-12 05:17:22.580 UTC [31559] LOG: database system is ready to accept connections
I20260812 05:17:23.586309 19099 mini_postgres.cc:96] Postgres bound to 38315
2026-08-12 05:17:23.592 UTC [31570] FATAL: database "slave" does not exist
127.18.166.212:38315 - accepting connections
I20260812 05:17:23.593458 19099 mini_ranger.cc:161] Starting Ranger...
I20260812 05:17:23.609525 19099 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 05:17:24.116814 19099 mini_ranger.cc:87] Created ranger Postgres database
I20260812 05:17:24.116914 19099 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-admin
2026-08-12 05:17:24,402 [I] DB FLAVOR :POSTGRES
2026-08-12 05:17:24,403 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:17:24,403 [I] Checking connection..
2026-08-12 05:17:24,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 05:17:24,677 [I] Checking connection passed.
2026-08-12 05:17:24,677 [I] --------- Verifying version history table ---------
2026-08-12 05:17:24,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:17:24,946 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:17:24,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:17:25,199 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:17:25,199 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 05:17:25,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 05:17:25.452 UTC [31637] WARNING: there is no transaction in progress
2026-08-12 05:17:25,458 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 05:17:25,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:17:25,726 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 05:17:25,726 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:17:25,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 05:17:25,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 05:17:26,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-srgv.c.gcp-upstream.internal','N') ;"
2026-08-12 05:17:26,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:17:26,751 [I] Table x_portal_user does not exist in database ranger
2026-08-12 05:17:26,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:17:26,999 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 05:17:27,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:17:27,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:17:27,507 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 05:17:27,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 05:17:28.505 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.524 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.542 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.559 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.578 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.597 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.658 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.665 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.680 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.691 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.703 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.718 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.729 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.735 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.743 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:28.750 UTC [31754] WARNING: there is no transaction in progress
2026-08-12 05:17:29,419 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 05:17:29,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 05:17:29,682 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 05:17:29,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:17:29,926 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 05:17:29,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:17:30,205 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 05:17:30,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:17:30,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:17:30,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-srgv.c.gcp-upstream.internal';"
2026-08-12 05:17:30,976 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 05:17:30,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 05:17:31,243 [I] DB_PATCHES have already been applied
I20260812 05:17:31.249286 19099 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-admin:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskjJH1Yp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 05:17:31.249383 19099 mini_ranger.cc:193] Using host: 127.18.166.212
I20260812 05:17:31.252864 19099 mini_ranger.cc:227] Ranger admin URL: http://127.18.166.212:50093
Aug 12, 2026 5:17:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:17:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:17:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 5:17:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 5:17:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 5:17:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 5:17:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 05:17:32.415522 19099 mini_ranger.cc:160] Time spent starting Ranger: real 8.822s user 0.000s sys 0.010s
Aug 12, 2026 5:17:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 5:17:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 5:17:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 5:17:52 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 5:17:53 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 05:17:54.113732 19099 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.166.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.166.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/rangerkms_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.166.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/rangerkms_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.166.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.166.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.166.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/HTTP_127.18.166.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 05:17:54 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511874, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 05:17:54.214745 19099 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 05:17:54.225071 19099 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 05:17:55.158232 19099 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 05:17:55.158334 19099 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms
I20260812 05:17:55.158360 19099 mini_ranger_kms.cc:227] Using postgres at 127.18.166.212:38315
2026-08-12 05:17:55,440 [I] DB FLAVOR :POSTGRES
2026-08-12 05:17:55,441 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:17:55,441 [I] Checking connection
2026-08-12 05:17:55,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 05:17:55,859 [I] connection success
2026-08-12 05:17:55,860 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 05:17:55,860 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 05:17:55,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 05:17:56,192 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 05:17:56,192 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:17:56,192 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 05:17:56,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.166.212:38315/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 05:17:56,470 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 05:17:56.971755 19099 mini_ranger_kms.cc:310] Created kms service
I20260812 05:17:57.156821 19099 mini_ranger_kms.cc:326] Created kudu user
I20260812 05:17:57.201926 19099 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 05:17:57 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511874, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:17:57.613603 19099 mini_ranger_kms.cc:384] Added ranger policy
I20260812 05:17:57.613739 19099 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms:/tmp/dist-test-taskjJH1Yp/build/release/bin/postgresql.jar:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskjJH1Yp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 05:17:57.613758 19099 mini_ranger_kms.cc:240] Using host: 127.18.166.212
I20260812 05:17:57.617475 19099 mini_ranger_kms.cc:276] Ranger KMS PID: 31949
I20260812 05:17:57.617560 19099 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.166.212:41217
Aug 12, 2026 5:17:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:17:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:17:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 5:17:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 5:17:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 5:17:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 05:17:59.955709 19099 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.741s user 0.009s sys 0.236s
I20260812 05:17:59.955812 19099 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 05:17:59.955861 19099 mini_ranger_kms.cc:401] 127.18.166.212:41217/kms/v1/keys
Aug 12 05:18:00 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786511880, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.166.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 05:18:00 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511880, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.166.212@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
Aug 12 05:18:01 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511880, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.166.212@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
Aug 12 05:18:03 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:18:03.424526 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43969
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:43969
--ranger_config_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:03.507124 32003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:03.507386 32003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:03.507426 32003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:03.508636 32003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:03.508704 32003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:03.508728 32003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:03.508750 32003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:03.508777 32003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:03.510447 32003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.166.212:41217/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:43969
--ranger_config_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:03.510730 32003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:03.510960 32003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:03.516096 32011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:03.516094 32009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:03.516090 32008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:03.516469 32003 server_base.cc:1034] running on GCE node
I20260812 05:18:03.516714 32003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:03.517036 32003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:03.518203 32003 hybrid_clock.cc:648] HybridClock initialized: now 1786511883518181 us; error 43 us; skew 500 ppm
Aug 12 05:18:03 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511883, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:03.520886 32003 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:03.521346 32003 webserver.cc:460] Webserver started at http://127.18.166.254:42957/ using document root <none> and password file <none>
I20260812 05:18:03.521595 32003 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:03.521659 32003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:03.521828 32003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:18:03 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511883, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:03.568629 32003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "0494a104ab3047b79d1a55c584aba990"
format_stamp: "Formatted at 2026-08-12 05:18:03 on dist-test-slave-srgv"
server_key: "f21f67253c681cb8024d0fdfb7b0ffb8"
server_key_iv: "2ef9d64947fb19f6203c1214c49314fa"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:03.569087 32003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "0494a104ab3047b79d1a55c584aba990"
format_stamp: "Formatted at 2026-08-12 05:18:03 on dist-test-slave-srgv"
server_key: "f21f67253c681cb8024d0fdfb7b0ffb8"
server_key_iv: "2ef9d64947fb19f6203c1214c49314fa"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:03.570740 32003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:18:03.583590 32018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:03.583886 32003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:03.584026 32003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "0494a104ab3047b79d1a55c584aba990"
format_stamp: "Formatted at 2026-08-12 05:18:03 on dist-test-slave-srgv"
server_key: "f21f67253c681cb8024d0fdfb7b0ffb8"
server_key_iv: "2ef9d64947fb19f6203c1214c49314fa"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:03.584157 32003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:03.602540 32003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:03.604862 32003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:03.605044 32003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:03.610193 32003 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:43969
I20260812 05:18:03.610291 32074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:43969 every 8 connection(s)
I20260812 05:18:03.610658 32003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:03.611021 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 32003
I20260812 05:18:03.611122 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:18:03.611296 19099 external_mini_cluster.cc:1428] Setting key d8354d0f16423692286725f59d9ad592
I20260812 05:18:03.611548 32075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:03.615370 32075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Bootstrap starting.
I20260812 05:18:03.616175 32075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:03.616500 32075 log.cc:824] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:03.617389 32075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: No bootstrap required, opened a new log
Aug 12 05:18:03 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:03.620910 32075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:03.621168 32075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:03.621258 32075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0494a104ab3047b79d1a55c584aba990, State: Initialized, Role: FOLLOWER
I20260812 05:18:03.621426 32075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:03.621515 32075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:03.621546 32075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:03.621593 32075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:03.622231 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:03.612001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.18.166.254:43969)
0812 05:18:03.612181 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:03.612190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:03.612222 (+ 32us) server_negotiation.cc:407] Connection header received
0812 05:18:03.612741 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:03.612747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:03.612794 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:03.612950 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:03.613649 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:03.614553 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:03.615319 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:03.615826 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:03.615951 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:03.618810 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:03.618814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:03.618814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:03.618827 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:03.620509 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:03.620896 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:03.620899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:03.620900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:03.621027 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:03.621234 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:03.621235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:03.621235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:03.621618 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:03.621795 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:03.621867 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 05:18:03.621988 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260812 05:18:03.622381 32075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:03.622494 32075 leader_election.cc:302] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0494a104ab3047b79d1a55c584aba990; no voters:
I20260812 05:18:03.622689 32075 leader_election.cc:288] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:18:03.622743 32080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:03.622992 32080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 LEADER]: Becoming Leader. State: Replica: 0494a104ab3047b79d1a55c584aba990, State: Running, Role: LEADER
I20260812 05:18:03.623068 32075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:03.623129 32080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:03.623644 32080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0494a104ab3047b79d1a55c584aba990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } } }
I20260812 05:18:03.623723 32080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:03.623847 32080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0494a104ab3047b79d1a55c584aba990. Latest consensus state: current_term: 1 leader_uuid: "0494a104ab3047b79d1a55c584aba990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } } }
I20260812 05:18:03.623930 32080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:03.624127 32090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:03.624718 32090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:03.625005 32075 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 05:18:03.629873 32090 catalog_manager.cc:1347] Generated new cluster ID: 2ebda37bf7594e4793cde10e3ac99e35
I20260812 05:18:03.629978 32090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:03.635597 32090 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:18:03.636560 32090 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:03.640817 32090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Generated new TSK 0
I20260812 05:18:03.641093 32090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:18:04.437045 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43969
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:04.527333 32125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:04.527601 32125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:04.527647 32125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:04.528898 32125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:04.528968 32125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:04.529017 32125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:04.530977 32125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.166.212:41217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:04.531277 32125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:04.531528 32125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:04.537842 32132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:04.537842 32134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:04.538206 32131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:04.539117 32125 server_base.cc:1034] running on GCE node
I20260812 05:18:04.539379 32125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:04.539657 32125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:04.540838 32125 hybrid_clock.cc:648] HybridClock initialized: now 1786511884540821 us; error 36 us; skew 500 ppm
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:04.544450 32125 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:04.545104 32125 webserver.cc:460] Webserver started at http://127.18.166.193:43765/ using document root <none> and password file <none>
I20260812 05:18:04.545445 32125 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:04.545573 32125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:04.545742 32125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:04.560267 32125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "29cff78f5a5348a491160b3b3e86309d"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "ad379e4922f60cadf88bccf9407586a7"
server_key_iv: "6a33d0af74e10707e1b1c08fe3d823d2"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:04.560839 32125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "29cff78f5a5348a491160b3b3e86309d"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "ad379e4922f60cadf88bccf9407586a7"
server_key_iv: "6a33d0af74e10707e1b1c08fe3d823d2"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:04.565835 32125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260812 05:18:04.579279 32142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:04.579468 32125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:04.579589 32125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "29cff78f5a5348a491160b3b3e86309d"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "ad379e4922f60cadf88bccf9407586a7"
server_key_iv: "6a33d0af74e10707e1b1c08fe3d823d2"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:04.579707 32125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:04.596709 32125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:04.599350 32125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:04.599576 32125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:04.599922 32125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:04.600382 32125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:04.600490 32125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:04.600600 32125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:04.600690 32125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:04.608163 32125 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:37909
I20260812 05:18:04.608678 32125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:18:04.609194 32264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:37909 every 8 connection(s)
I20260812 05:18:04.613505 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 32125
I20260812 05:18:04.613615 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:18:04.613813 19099 external_mini_cluster.cc:1428] Setting key 871db46308dc2687d2a1e6d36a5fac8d
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:04.639376 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:04.609647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43969 (local address 127.18.166.193:42705)
0812 05:18:04.617540 (+ 7893us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:04.617617 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:04.618121 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:04.621636 (+ 3515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:04.621642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:04.621944 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:04.622251 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:04.622257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.623397 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.623398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:04.624819 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:04.624823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.625430 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.625432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:04.625597 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:04.626277 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:04.626288 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:04.634722 (+ 8434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:04.637480 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:04.637493 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:04.637497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:04.637871 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:04.638152 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:04.638153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:04.638153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:04.638315 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:04.638643 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:04.638651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:04.638837 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 05:18:04.638944 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 05:18:04.639119 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7825,"thread_start_us":88,"threads_started":1}
I20260812 05:18:04.639868 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:04.609586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:42705 (local address 127.18.166.254:43969)
0812 05:18:04.621288 (+ 11702us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:04.621291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:04.621304 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:18:04.621335 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:04.621336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:04.621352 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:04.622364 (+ 1012us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:04.622409 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.623263 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.624947 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.625275 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.625351 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:04.634904 (+ 9553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:04.634907 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:04.634907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:04.634919 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:04.637307 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:04.637996 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:04.637998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:04.637999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:04.638076 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:04.638391 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:04.638392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:04.638393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:04.638544 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:04.638604 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:04.639671 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0812 05:18:04.639770 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11656,"thread_start_us":92,"threads_started":1}
I20260812 05:18:04.640475 32265 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43969
I20260812 05:18:04.640583 32265 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:04.640790 32265 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:04.641450 32038 ts_manager.cc:194] Registered new tserver with Master: 29cff78f5a5348a491160b3b3e86309d (127.18.166.193:37909)
I20260812 05:18:04.642501 32038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:42705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:18:04.657760 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43969
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:04.764026 32273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:04.764292 32273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:04.764334 32273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:04.765604 32273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:04.765666 32273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:04.765709 32273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:04.767196 32273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.166.212:41217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:04.767467 32273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:04.767710 32273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:04.772795 32281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:04.772894 32284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:04.772766 32280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:04.773787 32273 server_base.cc:1034] running on GCE node
I20260812 05:18:04.774039 32273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:04.774343 32273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:04.775489 32273 hybrid_clock.cc:648] HybridClock initialized: now 1786511884775481 us; error 52 us; skew 500 ppm
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:04.778302 32273 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:04.778776 32273 webserver.cc:460] Webserver started at http://127.18.166.194:36971/ using document root <none> and password file <none>
I20260812 05:18:04.779013 32273 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:04.779081 32273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:04.779224 32273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:04.796214 32273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "c00cebb11f48403b8d0dda3bf57bea31"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "df9f1d5e8f16dc95946f7eadd3a17429"
server_key_iv: "91f7ced25f43e5c6d1efc1f25cd93a88"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:04.796844 32273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "c00cebb11f48403b8d0dda3bf57bea31"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "df9f1d5e8f16dc95946f7eadd3a17429"
server_key_iv: "91f7ced25f43e5c6d1efc1f25cd93a88"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:04.805263 32273 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:04.837571 32293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:04.837817 32273 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.001s sys 0.001s
I20260812 05:18:04.837935 32273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "c00cebb11f48403b8d0dda3bf57bea31"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "df9f1d5e8f16dc95946f7eadd3a17429"
server_key_iv: "91f7ced25f43e5c6d1efc1f25cd93a88"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:04.838042 32273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:04.871399 32273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:04.874094 32273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:04.874331 32273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:04.874641 32273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:04.875097 32273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:04.875197 32273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:04.875279 32273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:04.875345 32273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:04.884318 32273 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:34009
I20260812 05:18:04.884892 32273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:18:04.885335 32415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:34009 every 8 connection(s)
I20260812 05:18:04.887509 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 32273
I20260812 05:18:04.887616 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:18:04.887866 19099 external_mini_cluster.cc:1428] Setting key f5b53774a53cf6bfbe455487f98b5e03
Aug 12 05:18:04 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:04.898017 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:04.886563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:57777 (local address 127.18.166.254:43969)
0812 05:18:04.886657 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:04.886660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:04.887142 (+ 482us) server_negotiation.cc:407] Connection header received
0812 05:18:04.887928 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:04.887930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:04.887948 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:04.887980 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:04.888866 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.889673 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.891199 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.891509 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.891555 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:04.894453 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:04.894454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:04.894454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:04.894462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:04.896280 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:04.896837 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:04.896838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:04.896839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:04.896902 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:04.897187 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:04.897188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:04.897189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:04.897328 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:04.897373 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:04.897817 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 05:18:04.897915 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 05:18:04.898813 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:04.886258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43969 (local address 127.18.166.194:57777)
0812 05:18:04.887001 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:04.887048 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:04.887802 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:04.888179 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:04.888184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:04.888446 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:04.888750 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:04.888756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.889799 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.889800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:04.891090 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:04.891093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:04.891682 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:04.891683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:04.891826 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:04.892491 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:04.892501 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:04.894336 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:04.896384 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:04.896392 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:04.896394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:04.896739 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:04.896972 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:04.896973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:04.896973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:04.897108 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:04.897457 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:04.897464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:04.897646 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 05:18:04.898385 (+ 739us) client_negotiation.cc:240] Negotiation successful
0812 05:18:04.898558 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":93,"threads_started":1}
I20260812 05:18:04.899499 32417 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43969
I20260812 05:18:04.899595 32417 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:04.899799 32417 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:04.900230 32038 ts_manager.cc:194] Registered new tserver with Master: c00cebb11f48403b8d0dda3bf57bea31 (127.18.166.194:34009)
I20260812 05:18:04.900718 32038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:57777
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:18:04.932847 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43969
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:05.076037 32425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:05.076432 32425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:05.076650 32425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:05.078584 32425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:05.078713 32425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:05.078807 32425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:05.081509 32425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.166.212:41217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:05.082019 32425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:05.082414 32425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:05.091424 32431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.091578 32430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.092448 32433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:05.093070 32425 server_base.cc:1034] running on GCE node
I20260812 05:18:05.093499 32425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:05.093837 32425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:05.095011 32425 hybrid_clock.cc:648] HybridClock initialized: now 1786511885094988 us; error 36 us; skew 500 ppm
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:05.099400 32425 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:05.099860 32425 webserver.cc:460] Webserver started at http://127.18.166.195:38675/ using document root <none> and password file <none>
I20260812 05:18:05.100179 32425 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:05.100262 32425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:05.100409 32425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:05.116899 32425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "6e171787ed9642f2bf7aa05245604b5a"
format_stamp: "Formatted at 2026-08-12 05:18:05 on dist-test-slave-srgv"
server_key: "0df85e17a5174bd0ffd528fa5ee05c5f"
server_key_iv: "000e51556adb0167df4eb930874535a4"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:05.118484 32425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "6e171787ed9642f2bf7aa05245604b5a"
format_stamp: "Formatted at 2026-08-12 05:18:05 on dist-test-slave-srgv"
server_key: "0df85e17a5174bd0ffd528fa5ee05c5f"
server_key_iv: "000e51556adb0167df4eb930874535a4"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:05.120771 32425 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 05:18:05.142874 32440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.143128 32425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:18:05.143231 32425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "6e171787ed9642f2bf7aa05245604b5a"
format_stamp: "Formatted at 2026-08-12 05:18:05 on dist-test-slave-srgv"
server_key: "0df85e17a5174bd0ffd528fa5ee05c5f"
server_key_iv: "000e51556adb0167df4eb930874535a4"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:05.143332 32425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:05.181222 32425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:05.184356 32425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:05.184557 32425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:05.185151 32425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:05.185742 32425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:05.185801 32425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.185837 32425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:05.185911 32425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.210055 32425 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43133
I20260812 05:18:05.210172 32567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43133 every 8 connection(s)
I20260812 05:18:05.210928 32425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:18:05.211778 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 32425
I20260812 05:18:05.211870 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:18:05.212071 19099 external_mini_cluster.cc:1428] Setting key 27d2743d8f3d61fad5ff02d074ca7675
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:05.237890 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:05.212810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43969 (local address 127.18.166.195:42019)
0812 05:18:05.213296 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:05.213345 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:05.214348 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:05.214738 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:05.214743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:05.215066 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:05.215454 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:05.215461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.217081 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.217082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:05.218445 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:05.218448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.219356 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.219357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:05.219554 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:05.220652 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:05.220663 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:05.223669 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:05.235804 (+ 12135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:05.235822 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:05.235825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:05.236221 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:05.236480 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:05.236481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:05.236481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:05.236647 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:05.236984 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:05.236991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:05.237202 (+ 211us) client_negotiation.cc:769] Sending connection context
0812 05:18:05.237313 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 05:18:05.237534 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":165,"threads_started":1}
I20260812 05:18:05.238513 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:05.212918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:42019 (local address 127.18.166.254:43969)
0812 05:18:05.212989 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:05.212992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:05.213471 (+ 479us) server_negotiation.cc:407] Connection header received
0812 05:18:05.214506 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:05.214507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:05.214540 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:05.214655 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:05.215580 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.216930 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.218566 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.219111 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.219293 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:05.223831 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:05.223834 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:05.223834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:05.223844 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:05.235657 (+ 11813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:05.236328 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:05.236330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:05.236330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:05.236405 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:05.236732 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:05.236733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:05.236734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:05.236890 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:05.236953 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:05.238311 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0812 05:18:05.238416 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:18:05.239471 32568 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43969
I20260812 05:18:05.239598 32568 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.239825 32568 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:05.240259 32038 ts_manager.cc:194] Registered new tserver with Master: 6e171787ed9642f2bf7aa05245604b5a (127.18.166.195:43133)
I20260812 05:18:05.240751 32038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:42019
I20260812 05:18:05.244890 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:18:05.245788 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 32125
I20260812 05:18:05.258687 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 32273
I20260812 05:18:05.276198 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 32425
I20260812 05:18:05.284442 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 32003
I20260812 05:18:05.290388 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43969
--webserver_interface=127.18.166.254
--webserver_port=42957
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:43969
--ranger_config_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:05.376252 32571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:05.376518 32571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:05.376547 32571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:05.377739 32571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:05.377779 32571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:05.377794 32571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:05.377806 32571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:05.377818 32571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:05.379410 32571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:43969
--ranger_config_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:43969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:05.379629 32571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:05.379808 32571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:05.384724 32580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.384733 32576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.384733 32577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:05.385116 32571 server_base.cc:1034] running on GCE node
I20260812 05:18:05.385432 32571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:05.385771 32571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:05.386945 32571 hybrid_clock.cc:648] HybridClock initialized: now 1786511885386937 us; error 38 us; skew 500 ppm
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:05.389657 32571 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:05.390196 32571 webserver.cc:460] Webserver started at http://127.18.166.254:42957/ using document root <none> and password file <none>
I20260812 05:18:05.390431 32571 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:05.390511 32571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:05.441231 32571 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260812 05:18:05.442189 32588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.442466 32571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:18:05.442554 32571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "0494a104ab3047b79d1a55c584aba990"
format_stamp: "Formatted at 2026-08-12 05:18:03 on dist-test-slave-srgv"
server_key: "f21f67253c681cb8024d0fdfb7b0ffb8"
server_key_iv: "2ef9d64947fb19f6203c1214c49314fa"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:05.442965 32571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:05.463897 32571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:05.465817 32571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:05.465992 32571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:05.470398 32571 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:43969
I20260812 05:18:05.470652 32647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:43969 every 8 connection(s)
I20260812 05:18:05.470963 32571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:05.472268 32648 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:18:05.472923 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Bootstrap starting.
I20260812 05:18:05.476783 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 32571
I20260812 05:18:05.477120 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:37909
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:43969
--webserver_port=43765
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:05.482175 32648 log.cc:824] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:05.483955 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:05.484290 32648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Bootstrap complete.
I20260812 05:18:05.486789 32648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:05.487025 32648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0494a104ab3047b79d1a55c584aba990, State: Initialized, Role: FOLLOWER
I20260812 05:18:05.487207 32648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:05.487286 32648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:05.487346 32648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:05.487394 32648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:18:05.488440 32648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:05.488528 32648 leader_election.cc:302] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0494a104ab3047b79d1a55c584aba990; no voters:
I20260812 05:18:05.488737 32648 leader_election.cc:288] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:18:05.488811 32652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:18:05.489008 32648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:05.489027 32652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [term 2 LEADER]: Becoming Leader. State: Replica: 0494a104ab3047b79d1a55c584aba990, State: Running, Role: LEADER
I20260812 05:18:05.489274 32652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } }
I20260812 05:18:05.489586 32653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0494a104ab3047b79d1a55c584aba990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } } }
I20260812 05:18:05.489657 32653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:05.489951 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0494a104ab3047b79d1a55c584aba990. Latest consensus state: current_term: 2 leader_uuid: "0494a104ab3047b79d1a55c584aba990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0494a104ab3047b79d1a55c584aba990" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 43969 } } }
I20260812 05:18:05.490000 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:05.490258 32661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:05.490721 32648 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 05:18:05.490839 32661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:05.491004 32661 catalog_manager.cc:1259] Loaded cluster ID: 2ebda37bf7594e4793cde10e3ac99e35
I20260812 05:18:05.491164 32661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:05.495677 32661 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:05.496086 32661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Loaded TSK: 0
I20260812 05:18:05.496425 32661 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:18:05.593007 32650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:05.593317 32650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:05.593361 32650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:05.595428 32650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:05.595606 32650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:05.595681 32650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:05.597966 32650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:37909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=43765
--tserver_master_addrs=127.18.166.254:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:05.598358 32650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:05.598668 32650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:05.605325 32692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.605482 32688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.605908 32689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:05.606823 32650 server_base.cc:1034] running on GCE node
I20260812 05:18:05.607055 32650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:05.607375 32650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:05.608580 32650 hybrid_clock.cc:648] HybridClock initialized: now 1786511885608562 us; error 38 us; skew 500 ppm
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:05.611910 32650 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:05.612505 32650 webserver.cc:460] Webserver started at http://127.18.166.193:43765/ using document root <none> and password file <none>
I20260812 05:18:05.612811 32650 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:05.612910 32650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:05.633667 32650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:05.634444 32700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.634653 32650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:18:05.634739 32650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "29cff78f5a5348a491160b3b3e86309d"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "ad379e4922f60cadf88bccf9407586a7"
server_key_iv: "6a33d0af74e10707e1b1c08fe3d823d2"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:05.635962 32650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:05.665807 32650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:05.668316 32650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:05.668542 32650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:05.668851 32650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:05.669245 32650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:05.669332 32650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.669453 32650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:05.669530 32650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.676054 32650 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:37909
I20260812 05:18:05.676620 32650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:18:05.679541 353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:37909 every 8 connection(s)
I20260812 05:18:05.685297 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 32650
I20260812 05:18:05.685664 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:34009
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:43969
--webserver_port=36971
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:05.694092 356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:05.680284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43969 (local address 127.18.166.193:52557)
0812 05:18:05.681077 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:05.681138 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:05.681898 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:05.682673 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:05.682677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:05.682929 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:05.683187 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:05.683192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.684826 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.684827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:05.686378 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:05.686381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.687292 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.687293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:05.687413 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:05.688030 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:05.688040 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:05.689851 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:05.692159 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:05.692167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:05.692169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:05.692450 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:05.692731 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:05.692731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:05.692732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:05.692854 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:05.693312 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:05.693321 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:05.693538 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 05:18:05.693662 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 05:18:05.693843 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":90,"threads_started":1}
I20260812 05:18:05.694746 358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:05.680697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:52557 (local address 127.18.166.254:43969)
0812 05:18:05.682122 (+ 1425us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:05.682132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:05.682170 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:18:05.682317 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:05.682323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:05.682379 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:05.682508 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:05.683316 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.684698 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.686479 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.687086 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.687180 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:05.689995 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:05.690001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:05.690002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:05.690017 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:05.692058 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:05.692541 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:05.692544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:05.692544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:05.692664 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:05.692918 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:05.692919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:05.692919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:05.693204 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:05.693292 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:05.694401 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0812 05:18:05.694539 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":81,"threads_started":1}
I20260812 05:18:05.695820 354 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43969
I20260812 05:18:05.695935 354 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.696157 354 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
W20260812 05:18:05.697220 354 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:43969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 05:18:05.697306 354 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.697388 354 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:05.697865 354 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.697948 354 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
W20260812 05:18:05.698354 354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:18:05.800863 359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:05.801167 359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:05.801236 359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:05.802597 359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:05.802665 359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:05.802712 359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:05.804347 359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:34009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=36971
--tserver_master_addrs=127.18.166.254:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:05.804636 359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:05.804900 359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:05.810424 368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:05.810410 365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:05.810638 359 server_base.cc:1034] running on GCE node
W20260812 05:18:05.810410 366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:05.810926 359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:05.811226 359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:05.812410 359 hybrid_clock.cc:648] HybridClock initialized: now 1786511885812358 us; error 47 us; skew 500 ppm
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:05.815300 359 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:05.815842 359 webserver.cc:460] Webserver started at http://127.18.166.194:36971/ using document root <none> and password file <none>
I20260812 05:18:05.816118 359 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:05.816208 359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:05.830893 359 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 05:18:05.832274 376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.832504 359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:18:05.832644 359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "c00cebb11f48403b8d0dda3bf57bea31"
format_stamp: "Formatted at 2026-08-12 05:18:04 on dist-test-slave-srgv"
server_key: "df9f1d5e8f16dc95946f7eadd3a17429"
server_key_iv: "91f7ced25f43e5c6d1efc1f25cd93a88"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:05.833146 359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:05.850688 359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:05.853068 359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:05.853241 359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:05.853529 359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:05.853864 359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:05.853935 359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.854007 359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:05.854036 359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:05.861682 359 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:34009
I20260812 05:18:05.861810 496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:34009 every 8 connection(s)
I20260812 05:18:05.862259 359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:18:05 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:05.870070 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 359
I20260812 05:18:05.870405 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.166.212:41217/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:43133
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:43969
--webserver_port=38675
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:05.872473 358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:05.863117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:40391 (local address 127.18.166.254:43969)
0812 05:18:05.863168 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:05.863170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:05.863423 (+ 253us) server_negotiation.cc:407] Connection header received
0812 05:18:05.864099 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:05.864100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:05.864113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:05.864170 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:05.864846 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.865334 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.866832 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.867022 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.867080 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:05.869453 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:05.869455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:05.869455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:05.869463 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:05.870917 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:05.871401 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:05.871403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:05.871403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:05.871464 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:05.871739 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:05.871740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:05.871740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:05.871854 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:05.871926 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:05.872256 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 05:18:05.872352 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:18:05.873067 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:05.863118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43969 (local address 127.18.166.194:40391)
0812 05:18:05.863284 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:05.863331 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:05.863959 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:05.864249 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:05.864254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:05.864487 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:05.864753 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:05.864759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.865452 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.865453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:05.866742 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:05.866746 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:05.867134 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:05.867135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:05.867294 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:05.867842 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:05.867852 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:05.869321 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:05.871022 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:05.871030 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:05.871032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:05.871314 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:05.871536 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:05.871537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:05.871538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:05.871661 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:05.871917 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:05.871924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:05.872073 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 05:18:05.872767 (+ 694us) client_negotiation.cc:240] Negotiation successful
0812 05:18:05.872900 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260812 05:18:05.873588 497 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43969
I20260812 05:18:05.873713 497 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.873898 497 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
W20260812 05:18:05.874537 497 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:43969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 05:18:05.874627 497 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.874702 497 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:05.874995 497 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:05.875078 497 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
W20260812 05:18:05.875347 497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:18:05.996793 500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:05.997116 500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:05.997159 500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:05.998965 500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:05.999112 500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:05.999214 500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:06.001940 500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.166.212:41217/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:43133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=38675
--tserver_master_addrs=127.18.166.254:43969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:06.002480 500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:06.002802 500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:06.008951 508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:06.009158 507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:06.008937 510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:06.009627 500 server_base.cc:1034] running on GCE node
I20260812 05:18:06.009863 500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:06.010141 500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:06.011314 500 hybrid_clock.cc:648] HybridClock initialized: now 1786511886011295 us; error 38 us; skew 500 ppm
Aug 12 05:18:06 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:06.018018 500 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:06.018654 500 webserver.cc:460] Webserver started at http://127.18.166.195:38675/ using document root <none> and password file <none>
I20260812 05:18:06.018952 500 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:06.019037 500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:18:06 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:06.038585 500 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:06.041602 519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:06.041858 500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 05:18:06.041954 500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "6e171787ed9642f2bf7aa05245604b5a"
format_stamp: "Formatted at 2026-08-12 05:18:05 on dist-test-slave-srgv"
server_key: "0df85e17a5174bd0ffd528fa5ee05c5f"
server_key_iv: "000e51556adb0167df4eb930874535a4"
server_key_version: "kuduclusterkey@0"
I20260812 05:18:06.042461 500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:06.064649 500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:06.066931 500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:06.067144 500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:06.067476 500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:06.067905 500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:06.067999 500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:06.068051 500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:06.068071 500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:06.075941 500 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43133
I20260812 05:18:06.076436 500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:18:06.077878 640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43133 every 8 connection(s)
I20260812 05:18:06.078505 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 500
Aug 12 05:18:06 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:06.089641 358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:06.078876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:39083 (local address 127.18.166.254:43969)
0812 05:18:06.079084 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:06.079087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:06.079445 (+ 358us) server_negotiation.cc:407] Connection header received
0812 05:18:06.080004 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:06.080005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:06.080017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:06.080082 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:06.080870 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:06.081490 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:06.082700 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:06.082983 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:06.083046 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:06.085567 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:06.085568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:06.085568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:06.085582 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:06.087906 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:06.088417 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:06.088418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:06.088418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:06.088472 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:06.088737 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:06.088738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:06.088738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:06.088860 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:06.088937 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:06.089374 (+ 437us) server_negotiation.cc:299] Negotiation successful
0812 05:18:06.089512 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260812 05:18:06.089732 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:06.078944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.18.166.254:43969)
0812 05:18:06.079255 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:06.079257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:06.080229 (+ 972us) server_negotiation.cc:407] Connection header received
0812 05:18:06.080310 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:06.080311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:06.080322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:06.080392 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:06.081047 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:06.081819 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:06.083704 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:06.084025 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:06.084116 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:06.085699 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:06.085700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:06.085701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:06.085707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:06.087684 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:06.088106 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:06.088107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:06.088108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:06.088167 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:06.089054 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:06.089055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:06.089056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:06.089174 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:06.089341 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:06.089432 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 05:18:06.089604 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":291,"thread_start_us":73,"threads_started":1}
I20260812 05:18:06.090477 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:06.078560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:43969 (local address 127.18.166.195:39083)
0812 05:18:06.079312 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:06.079359 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:06.079884 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:06.080199 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:06.080204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:06.080457 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:06.080758 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:06.080764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:06.081598 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:06.081599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:06.082591 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:06.082594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:06.083136 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:06.083137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:06.083281 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:06.083944 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:06.083955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:06.085376 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:06.087981 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:06.087989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:06.087991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:06.088340 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:06.088525 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:06.088526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:06.088526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:06.088660 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:06.088929 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:06.088936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:06.089132 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 05:18:06.089974 (+ 842us) client_negotiation.cc:240] Negotiation successful
0812 05:18:06.090163 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":98,"threads_started":1}
I20260812 05:18:06.091064 641 heartbeater.cc:347] Connected to a master server at 127.18.166.254:43969
I20260812 05:18:06.091188 641 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:06.091378 641 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
W20260812 05:18:06.092036 641 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:43969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 05:18:06.092123 641 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:06.092190 641 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:06.092511 641 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:06.092587 641 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
W20260812 05:18:06.092864 641 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 05:18:06 dist-test-slave-srgv krb5kdc[31525](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786511886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:06.698558 354 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:06.698734 354 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:06.699167 32605 ts_manager.cc:194] Registered new tserver with Master: 29cff78f5a5348a491160b3b3e86309d (127.18.166.193:37909)
I20260812 05:18:06.700179 32605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:52557
I20260812 05:18:06.875550 497 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:06.875712 497 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:06.876068 32605 ts_manager.cc:194] Registered new tserver with Master: c00cebb11f48403b8d0dda3bf57bea31 (127.18.166.194:34009)
I20260812 05:18:06.876616 32605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:40391
Aug 12 05:18:06 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
I20260812 05:18:07.093080 641 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:07.093277 641 heartbeater.cc:511] Master 127.18.166.254:43969 requested a full tablet report, sending...
I20260812 05:18:07.093765 32605 ts_manager.cc:194] Registered new tserver with Master: 6e171787ed9642f2bf7aa05245604b5a (127.18.166.195:43133)
I20260812 05:18:07.094327 32605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:39083
I20260812 05:18:07.100003 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:18:07.109805 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.101141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.18.166.254:43969)
0812 05:18:07.101372 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.101374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.101529 (+ 155us) server_negotiation.cc:407] Connection header received
0812 05:18:07.101638 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.101638 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.101651 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.101697 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:07.102519 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.103333 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.104545 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.104851 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.104896 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.106589 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:07.106590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:07.106590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:07.106599 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:07.108499 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:07.108894 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:07.108895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:07.108896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:07.108956 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:07.109173 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:07.109174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:07.109174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:07.109303 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:07.109374 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.109613 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.109703 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":81,"threads_started":1}
I20260812 05:18:07.112637 32605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:18:07.115525 32605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:18:07.126489 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.120150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.18.166.193:37909)
0812 05:18:07.120281 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.120287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.121325 (+ 1038us) server_negotiation.cc:407] Connection header received
0812 05:18:07.121549 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.121553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.121572 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.121629 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:07.121979 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.123214 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.124883 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.125665 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.125730 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.125785 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.126343 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.126392 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":50,"threads_started":1}
I20260812 05:18:07.126986 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.120048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:37909 (local address 127.0.0.1:38612)
0812 05:18:07.121203 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.121253 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:07.121371 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:07.121815 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:07.121818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:07.121820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:07.121895 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.121897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.123595 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.123596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.124776 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.124779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.126136 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.126137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.126223 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.126238 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:18:07.126818 (+ 580us) client_negotiation.cc:240] Negotiation successful
0812 05:18:07.126902 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"thread_start_us":454,"threads_started":1}
I20260812 05:18:07.128072 32751 tablet_service.cc:1467] Processing CreateTablet for tablet 2680a3b5e4c14966b8d11f507e4b5cd7 (DEFAULT_TABLE table=test-table [id=3cb37d7d478041ab91b1c256dc672ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:07.128484 32751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2680a3b5e4c14966b8d11f507e4b5cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:07.132476 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.120169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.18.166.195:43133)
0812 05:18:07.120354 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.120358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.127387 (+ 7029us) server_negotiation.cc:407] Connection header received
0812 05:18:07.127504 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.127507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.127525 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.127583 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:07.127859 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.129252 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.130668 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.131555 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.131656 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.131713 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.131932 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.131979 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":53,"threads_started":1}
I20260812 05:18:07.132736 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.120061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43133 (local address 127.0.0.1:44680)
0812 05:18:07.127321 (+ 7260us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.127333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:07.127420 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:07.127749 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:07.127750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:07.127752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:07.127784 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.127786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.129357 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.129358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.130555 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.130558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.131777 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.131778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.131842 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.131845 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:18:07.132584 (+ 739us) client_negotiation.cc:240] Negotiation successful
0812 05:18:07.132651 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7247,"thread_start_us":314,"threads_started":1}
I20260812 05:18:07.133714 565 tablet_service.cc:1467] Processing CreateTablet for tablet 2680a3b5e4c14966b8d11f507e4b5cd7 (DEFAULT_TABLE table=test-table [id=3cb37d7d478041ab91b1c256dc672ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:07.134057 565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2680a3b5e4c14966b8d11f507e4b5cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:07.138244 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.120626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:34009 (local address 127.0.0.1:38022)
0812 05:18:07.132826 (+ 12200us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.132840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:07.132891 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:07.133448 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:07.133449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:07.133451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:07.133486 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.133487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.134764 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.134765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.135990 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.135994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.137872 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.137873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.137934 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.137938 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:07.138007 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:18:07.138089 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12184,"thread_start_us":120,"threads_started":1}
I20260812 05:18:07.138525 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.120744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.18.166.194:34009)
0812 05:18:07.133104 (+ 12360us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.133109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.133121 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:18:07.133156 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.133158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.133174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.133253 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:07.133574 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.134649 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.136602 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.137676 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.137758 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.137831 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.138376 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.138419 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12309,"thread_start_us":56,"threads_started":1}
I20260812 05:18:07.139746 427 tablet_service.cc:1467] Processing CreateTablet for tablet 2680a3b5e4c14966b8d11f507e4b5cd7 (DEFAULT_TABLE table=test-table [id=3cb37d7d478041ab91b1c256dc672ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:07.140091 678 tablet_bootstrap.cc:492] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: Bootstrap starting.
I20260812 05:18:07.140419 427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2680a3b5e4c14966b8d11f507e4b5cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:07.146018 678 tablet_bootstrap.cc:654] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:07.146898 678 log.cc:824] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:07.151389 680 tablet_bootstrap.cc:492] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Bootstrap starting.
I20260812 05:18:07.153493 680 tablet_bootstrap.cc:654] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:07.154389 678 tablet_bootstrap.cc:492] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: No bootstrap required, opened a new log
I20260812 05:18:07.154491 678 ts_tablet_manager.cc:1397] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260812 05:18:07.154563 680 log.cc:824] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:07.156826 681 tablet_bootstrap.cc:492] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Bootstrap starting.
I20260812 05:18:07.158567 681 tablet_bootstrap.cc:654] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:07.158560 678 raft_consensus.cc:348] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.158850 678 raft_consensus.cc:374] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:07.158932 678 raft_consensus.cc:729] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cff78f5a5348a491160b3b3e86309d, State: Initialized, Role: FOLLOWER
I20260812 05:18:07.159132 678 consensus_queue.cc:260] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.159545 678 ts_tablet_manager.cc:1428] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 05:18:07.159723 354 heartbeater.cc:503] Master 127.18.166.254:43969 was elected leader, sending a full tablet report...
I20260812 05:18:07.159633 681 log.cc:824] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:07.161353 681 tablet_bootstrap.cc:492] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: No bootstrap required, opened a new log
I20260812 05:18:07.161502 680 tablet_bootstrap.cc:492] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: No bootstrap required, opened a new log
I20260812 05:18:07.161517 681 ts_tablet_manager.cc:1397] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 05:18:07.161597 680 ts_tablet_manager.cc:1397] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260812 05:18:07.165979 681 raft_consensus.cc:348] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.167037 681 raft_consensus.cc:374] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:07.167132 681 raft_consensus.cc:729] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c00cebb11f48403b8d0dda3bf57bea31, State: Initialized, Role: FOLLOWER
I20260812 05:18:07.167333 681 consensus_queue.cc:260] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.167742 681 ts_tablet_manager.cc:1428] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260812 05:18:07.167904 497 heartbeater.cc:503] Master 127.18.166.254:43969 was elected leader, sending a full tablet report...
I20260812 05:18:07.174532 680 raft_consensus.cc:348] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.174899 680 raft_consensus.cc:374] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:07.175004 680 raft_consensus.cc:729] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e171787ed9642f2bf7aa05245604b5a, State: Initialized, Role: FOLLOWER
I20260812 05:18:07.175228 680 consensus_queue.cc:260] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.175699 680 ts_tablet_manager.cc:1428] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Time spent starting tablet: real 0.014s user 0.000s sys 0.006s
I20260812 05:18:07.175895 641 heartbeater.cc:503] Master 127.18.166.254:43969 was elected leader, sending a full tablet report...
W20260812 05:18:07.182045 355 tablet.cc:2369] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:18:07.330135 642 tablet.cc:2369] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 05:18:07 dist-test-slave-srgv krb5kdc[31525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for HTTP/127.18.166.212@KRBTEST.COM
W20260812 05:18:07.365067 498 tablet.cc:2369] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:07.439998 686 raft_consensus.cc:482] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:07.440248 686 raft_consensus.cc:504] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.440860 686 leader_election.cc:288] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29cff78f5a5348a491160b3b3e86309d (127.18.166.193:37909), c00cebb11f48403b8d0dda3bf57bea31 (127.18.166.194:34009)
I20260812 05:18:07.451128 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.441848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:49375 (local address 127.18.166.193:37909)
0812 05:18:07.441891 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.441893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.445973 (+ 4080us) server_negotiation.cc:407] Connection header received
0812 05:18:07.446626 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.446628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.446644 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.446727 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:07.447131 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.447819 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.449919 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.450545 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.450599 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.450610 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.450994 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.451047 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:18:07.451807 689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.441039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:37909 (local address 127.18.166.195:49375)
0812 05:18:07.445819 (+ 4780us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.445836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:07.446450 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:07.446794 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:07.446796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:07.446798 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:07.447028 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.447031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.447979 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.447980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.449803 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.449807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.450793 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.450794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.450899 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.450903 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:07.451493 (+ 590us) client_negotiation.cc:240] Negotiation successful
0812 05:18:07.451543 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4726,"thread_start_us":177,"threads_started":1}
I20260812 05:18:07.452566 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2680a3b5e4c14966b8d11f507e4b5cd7" candidate_uuid: "6e171787ed9642f2bf7aa05245604b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cff78f5a5348a491160b3b3e86309d" is_pre_election: true
I20260812 05:18:07.452766 303 raft_consensus.cc:2454] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e171787ed9642f2bf7aa05245604b5a in term 0.
I20260812 05:18:07.453094 523 leader_election.cc:302] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cff78f5a5348a491160b3b3e86309d, 6e171787ed9642f2bf7aa05245604b5a; no voters:
I20260812 05:18:07.453238 686 raft_consensus.cc:2781] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:18:07.453277 686 raft_consensus.cc:482] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:18:07.453300 686 raft_consensus.cc:3037] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:07.456223 686 raft_consensus.cc:504] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.456372 686 leader_election.cc:288] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [CANDIDATE]: Term 1 election: Requested vote from peers 29cff78f5a5348a491160b3b3e86309d (127.18.166.193:37909), c00cebb11f48403b8d0dda3bf57bea31 (127.18.166.194:34009)
I20260812 05:18:07.456663 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2680a3b5e4c14966b8d11f507e4b5cd7" candidate_uuid: "6e171787ed9642f2bf7aa05245604b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cff78f5a5348a491160b3b3e86309d"
I20260812 05:18:07.456760 303 raft_consensus.cc:3037] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:07.458035 303 raft_consensus.cc:2454] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e171787ed9642f2bf7aa05245604b5a in term 1.
I20260812 05:18:07.458407 523 leader_election.cc:302] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cff78f5a5348a491160b3b3e86309d, 6e171787ed9642f2bf7aa05245604b5a; no voters:
I20260812 05:18:07.458534 686 raft_consensus.cc:2781] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:07.458694 686 raft_consensus.cc:686] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 1 LEADER]: Becoming Leader. State: Replica: 6e171787ed9642f2bf7aa05245604b5a, State: Running, Role: LEADER
I20260812 05:18:07.458797 686 consensus_queue.cc:237] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } }
I20260812 05:18:07.465991 32605 catalog_manager.cc:5495] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e171787ed9642f2bf7aa05245604b5a (127.18.166.195). New cstate: current_term: 1 leader_uuid: "6e171787ed9642f2bf7aa05245604b5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e171787ed9642f2bf7aa05245604b5a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 } health_report { overall_health: UNKNOWN } } }
I20260812 05:18:07.470664 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.441449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:50679 (local address 127.18.166.194:34009)
0812 05:18:07.441728 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.441730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.452384 (+ 10654us) server_negotiation.cc:407] Connection header received
0812 05:18:07.452523 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.452524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.452546 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.452587 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:07.467392 (+ 14805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.468162 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.469584 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.470224 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.470277 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.470287 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.470538 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.470595 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260812 05:18:07.471076 689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.441356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:34009 (local address 127.18.166.195:50679)
0812 05:18:07.452296 (+ 10940us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.452309 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:07.452442 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:07.452661 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:07.452662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:07.452664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:07.452805 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.452807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.468277 (+ 15470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.468278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.469391 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:07.469395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.470346 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.470347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:07.470458 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.470462 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:07.470951 (+ 489us) client_negotiation.cc:240] Negotiation successful
0812 05:18:07.471003 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10925,"thread_start_us":158,"threads_started":1}
I20260812 05:18:07.471768 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2680a3b5e4c14966b8d11f507e4b5cd7" candidate_uuid: "6e171787ed9642f2bf7aa05245604b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c00cebb11f48403b8d0dda3bf57bea31" is_pre_election: true
I20260812 05:18:07.471969 449 raft_consensus.cc:2454] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e171787ed9642f2bf7aa05245604b5a in term 0.
I20260812 05:18:07.472136 448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2680a3b5e4c14966b8d11f507e4b5cd7" candidate_uuid: "6e171787ed9642f2bf7aa05245604b5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c00cebb11f48403b8d0dda3bf57bea31"
I20260812 05:18:07.472249 448 raft_consensus.cc:3037] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:07.473642 448 raft_consensus.cc:2454] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e171787ed9642f2bf7aa05245604b5a in term 1.
I20260812 05:18:07.478112 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.474770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.18.166.195:43133)
0812 05:18:07.474881 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:07.474883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:07.474891 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:18:07.474923 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:07.474924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:07.474943 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:07.475012 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:07.475196 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.475898 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.476982 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:07.477267 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:07.477324 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:07.477596 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:07.477722 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:18:07.477794 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:07.478016 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 05:18:07.478038 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 05:18:07.480696 448 raft_consensus.cc:1261] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 1 FOLLOWER]: Refusing update from remote peer 6e171787ed9642f2bf7aa05245604b5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:07.480865 303 raft_consensus.cc:1261] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 29cff78f5a5348a491160b3b3e86309d [term 1 FOLLOWER]: Refusing update from remote peer 6e171787ed9642f2bf7aa05245604b5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:07.481168 686 consensus_queue.cc:1035] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c00cebb11f48403b8d0dda3bf57bea31" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:07.481315 686 consensus_queue.cc:1035] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29cff78f5a5348a491160b3b3e86309d" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:07.493665 694 mvcc.cc:205] Tried to move back new op lower bound from 7317552691118120960 to 7317552691056517120. Current Snapshot: MvccSnapshot[applied={T|T < 7317552691118120960}]
I20260812 05:18:07.493975 697 mvcc.cc:205] Tried to move back new op lower bound from 7317552691118120960 to 7317552691056517120. Current Snapshot: MvccSnapshot[applied={T|T < 7317552691118120960}]
I20260812 05:18:07.496057 695 mvcc.cc:205] Tried to move back new op lower bound from 7317552691118120960 to 7317552691056517120. Current Snapshot: MvccSnapshot[applied={T|T < 7317552691118120960}]
I20260812 05:18:07.499506 32605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:07.499637 32605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:07.500576 32605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0494a104ab3047b79d1a55c584aba990: Sending DeleteTablet for 3 replicas of tablet 2680a3b5e4c14966b8d11f507e4b5cd7
I20260812 05:18:07.501205 32751 tablet_service.cc:1514] Processing DeleteTablet for tablet 2680a3b5e4c14966b8d11f507e4b5cd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:07 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:38612
I20260812 05:18:07.501231 427 tablet_service.cc:1514] Processing DeleteTablet for tablet 2680a3b5e4c14966b8d11f507e4b5cd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:07 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:38022
I20260812 05:18:07.501878 565 tablet_service.cc:1514] Processing DeleteTablet for tablet 2680a3b5e4c14966b8d11f507e4b5cd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:07 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44680
I20260812 05:18:07.502498 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 32650
I20260812 05:18:07.513377 705 tablet_replica.cc:330] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: stopping tablet replica
I20260812 05:18:07.513698 705 raft_consensus.cc:2229] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:07.513971 705 raft_consensus.cc:2258] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 05:18:07.514292 32592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:37909 (error 108)
I20260812 05:18:07.515178 705 ts_tablet_manager.cc:1905] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:07.517446 705 ts_tablet_manager.cc:1918] T 2680a3b5e4c14966b8d11f507e4b5cd7 P c00cebb11f48403b8d0dda3bf57bea31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:07.517498 706 tablet_replica.cc:330] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: stopping tablet replica
I20260812 05:18:07.517717 706 raft_consensus.cc:2229] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:18:07.517700 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 359
I20260812 05:18:07.517935 706 raft_consensus.cc:2258] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:07.519531 706 ts_tablet_manager.cc:1905] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:07.525987 706 ts_tablet_manager.cc:1918] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:07.526222 706 log.cc:1196] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/2680a3b5e4c14966b8d11f507e4b5cd7
I20260812 05:18:07.526602 706 ts_tablet_manager.cc:1939] T 2680a3b5e4c14966b8d11f507e4b5cd7 P 6e171787ed9642f2bf7aa05245604b5a: Deleting consensus metadata
I20260812 05:18:07.527236 32591 catalog_manager.cc:4841] TS 6e171787ed9642f2bf7aa05245604b5a (127.18.166.195:43133): tablet 2680a3b5e4c14966b8d11f507e4b5cd7 (table test-table [id=3cb37d7d478041ab91b1c256dc672ba7]) successfully deleted
I20260812 05:18:07.529973 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:07.529753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:37909 (local address 127.0.0.1:38618)
0812 05:18:07.529851 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.529931 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:37909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 05:18:07.530117 32592 catalog_manager.cc:4584] TS 29cff78f5a5348a491160b3b3e86309d (127.18.166.193:37909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2680a3b5e4c14966b8d11f507e4b5cd7: Network error: Client connection negotiation failed: client connection to 127.18.166.193:37909: connect: Connection refused (error 111)
W20260812 05:18:07.530815 675 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:18:07.530499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:34009 (local address 127.0.0.1:38034)
0812 05:18:07.530554 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:07.530565 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:07.530625 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:07.530768 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.194:34009: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 05:18:07.533727 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 500
I20260812 05:18:07.541013 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 32571
I20260812 05:18:07.549897 19099 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 05:18:07.894972 19099 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 05:18:07.895085 19099 mini_ranger.cc:66] Stopping Ranger...
I20260812 05:18:08.239995 19099 mini_ranger.cc:68] Stopped Ranger
2026-08-12 05:18:08.240 UTC [31559] LOG: received smart shutdown request
2026-08-12 05:18:08.244 UTC [31559] LOG: background worker "logical replication launcher" (PID 31568) exited with exit code 1
2026-08-12 05:18:08.245 UTC [31563] LOG: shutting down
2026-08-12 05:18:08.411 UTC [31559] LOG: database system is shut down
2026-08-12T05:18:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49803 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:18:08 dist-test-slave-srgv krb5kdc[716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:18:08 dist-test-slave-srgv krb5kdc[716](info): set up 2 sockets
Aug 12 05:18:08 dist-test-slave-srgv krb5kdc[716](info): commencing operation
krb5kdc: starting...
W20260812 05:18:09.602713 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:18:09 dist-test-slave-srgv krb5kdc[716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:18:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:18:09.728264 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:40001
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:40001,127.18.166.253:36035,127.18.166.252:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:09.812629 730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:09.812903 730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:09.812945 730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:09.814244 730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:09.814321 730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:09.814349 730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:09.814384 730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:09.814414 730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:09.816077 730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:40001,127.18.166.253:36035,127.18.166.252:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:40001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:09.816381 730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:09.816625 730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:09.821849 736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:09.821856 735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:09.821826 739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:09.822008 730 server_base.cc:1034] running on GCE node
I20260812 05:18:09.822355 730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:09.822682 730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:09.823866 730 hybrid_clock.cc:648] HybridClock initialized: now 1786511889823840 us; error 41 us; skew 500 ppm
Aug 12 05:18:09 dist-test-slave-srgv krb5kdc[716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:09.826524 730 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:09.827008 730 webserver.cc:460] Webserver started at http://127.18.166.254:45277/ using document root <none> and password file <none>
I20260812 05:18:09.827261 730 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:09.827333 730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:09.827492 730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:09.828641 730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "f47eb08ffd714a6793a85acd1d579a86"
format_stamp: "Formatted at 2026-08-12 05:18:09 on dist-test-slave-srgv"
server_key: "11f8d780a2d3579640e895b3c7b40eec"
server_key_iv: "a6d77d29120c70a806f884034bc609ea"
server_key_version: "encryptionkey@0"
I20260812 05:18:09.829066 730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "f47eb08ffd714a6793a85acd1d579a86"
format_stamp: "Formatted at 2026-08-12 05:18:09 on dist-test-slave-srgv"
server_key: "11f8d780a2d3579640e895b3c7b40eec"
server_key_iv: "a6d77d29120c70a806f884034bc609ea"
server_key_version: "encryptionkey@0"
I20260812 05:18:09.830621 730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:18:09.831449 746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:09.831640 730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:09.831756 730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "f47eb08ffd714a6793a85acd1d579a86"
format_stamp: "Formatted at 2026-08-12 05:18:09 on dist-test-slave-srgv"
server_key: "11f8d780a2d3579640e895b3c7b40eec"
server_key_iv: "a6d77d29120c70a806f884034bc609ea"
server_key_version: "encryptionkey@0"
I20260812 05:18:09.831849 730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:09.854915 730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:09.857195 730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:09.857391 730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:09.861716 730 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:40001
I20260812 05:18:09.861776 806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:40001 every 8 connection(s)
I20260812 05:18:09.862174 730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:09.863102 807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:09.863795 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 730
I20260812 05:18:09.863914 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:18:09.864117 19099 external_mini_cluster.cc:1428] Setting key 3bd2fdaa88f97dbc6ac2bf99ed9e24c6
I20260812 05:18:09.864904 807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:18:09 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.253@KRBTEST.COM" created.
I20260812 05:18:09.876577 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.865795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.18.166.254:40001)
0812 05:18:09.865998 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:09.866009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:09.866122 (+ 113us) server_negotiation.cc:407] Connection header received
0812 05:18:09.866682 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:09.866719 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:09.866738 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:09.866866 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:09.867683 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:09.868521 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:09.870120 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:09.870444 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:09.870530 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:09.873124 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:09.873131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:09.873131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:09.873138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:09.874769 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:09.875197 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:09.875200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:09.875201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:09.875404 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:09.875642 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:09.875642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:09.875643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:09.875870 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:09.875942 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:09.876239 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 05:18:09.876394 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260812 05:18:09.876739 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.865783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:40001 (local address 127.0.0.1:46830)
0812 05:18:09.865992 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.866031 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:09.866520 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:09.866902 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:09.866906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:09.867332 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:09.867569 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:09.867575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:09.868623 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:09.868625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:09.870016 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:09.870020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:09.870548 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:09.870550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:09.870636 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:09.871273 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:09.871284 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:09.872983 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:09.874853 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:09.874860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:09.874862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:09.875094 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:09.875480 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:09.875481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:09.875481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:09.875576 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:09.875960 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:09.875967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:09.876061 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 05:18:09.876175 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 05:18:09.876487 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":128,"thread_start_us":95,"threads_started":1}
I20260812 05:18:09.877545 807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:18:09.877934 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.877819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36046)
0812 05:18:09.877869 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.877915 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 05:18:09.878108 751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
I20260812 05:18:09.878619 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.878507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36048)
0812 05:18:09.878558 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.878591 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 05:18:09.878800 807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:36035: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab.
Entry for principal kudu/127.18.166.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab.
I20260812 05:18:09.893500 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:36035
--webserver_interface=127.18.166.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab
--principal=kudu/127.18.166.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:40001,127.18.166.253:36035,127.18.166.252:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:09.928056 807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } attempt: 1
I20260812 05:18:09.928700 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.928541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36064)
0812 05:18:09.928620 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.928667 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 05:18:09.929155 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.929026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36066)
0812 05:18:09.929107 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.929136 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 05:18:09.929284 807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:36035: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
W20260812 05:18:09.982306 815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:09.982532 815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:09.982556 815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:09.983754 815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:09.983803 815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:09.983817 815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:09.983830 815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:09.983887 815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:09.985390 815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:40001,127.18.166.253:36035,127.18.166.252:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab
--principal=kudu/127.18.166.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:36035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.166.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:09.985687 815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:09.985929 815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:09.991232 823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:09.991226 821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:09.991228 820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:09.991393 815 server_base.cc:1034] running on GCE node
I20260812 05:18:09.991701 815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:09.991997 815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:09.993151 815 hybrid_clock.cc:648] HybridClock initialized: now 1786511889993136 us; error 27 us; skew 500 ppm
Aug 12 05:18:09 dist-test-slave-srgv krb5kdc[716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:09.995944 815 init.cc:377] Logged in from keytab as kudu/127.18.166.253@KRBTEST.COM (short username kudu)
I20260812 05:18:09.996459 815 webserver.cc:460] Webserver started at http://127.18.166.253:33359/ using document root <none> and password file <none>
I20260812 05:18:09.996766 815 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:09.996832 815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:09.997004 815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:09.997478 807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } attempt: 2
I20260812 05:18:09.997872 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.997742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36078)
0812 05:18:09.997807 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.997844 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 05:18:09.998273 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:09.998182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36094)
0812 05:18:09.998232 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:09.998257 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 05:18:09.998399 807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:36035: Network error: Client connection negotiation failed: client connection to 127.18.166.253:36035: connect: Connection refused (error 111)
I20260812 05:18:09.998371 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data/instance:
uuid: "56fb9bd0196a4c38a5ceef64bb68e33c"
format_stamp: "Formatted at 2026-08-12 05:18:09 on dist-test-slave-srgv"
server_key: "ecc70635d37cf7ae7753bf6cebf02d5e"
server_key_iv: "e428e649ec1dfb269b66d86522d0869d"
server_key_version: "encryptionkey@0"
I20260812 05:18:09.998816 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal/instance:
uuid: "56fb9bd0196a4c38a5ceef64bb68e33c"
format_stamp: "Formatted at 2026-08-12 05:18:09 on dist-test-slave-srgv"
server_key: "ecc70635d37cf7ae7753bf6cebf02d5e"
server_key_iv: "e428e649ec1dfb269b66d86522d0869d"
server_key_version: "encryptionkey@0"
I20260812 05:18:10.000183 815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:18:10.001020 830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:10.001226 815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:10.001332 815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal
uuid: "56fb9bd0196a4c38a5ceef64bb68e33c"
format_stamp: "Formatted at 2026-08-12 05:18:09 on dist-test-slave-srgv"
server_key: "ecc70635d37cf7ae7753bf6cebf02d5e"
server_key_iv: "e428e649ec1dfb269b66d86522d0869d"
server_key_version: "encryptionkey@0"
I20260812 05:18:10.001446 815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:10.017037 815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:10.020102 815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:10.020303 815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:10.024897 815 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.253:36035
I20260812 05:18:10.024976 888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.253:36035 every 8 connection(s)
I20260812 05:18:10.025368 815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
I20260812 05:18:10.026362 890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:10.028085 890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:18:10.029518 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 815
I20260812 05:18:10.029639 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-1/wal/instance
I20260812 05:18:10.029855 19099 external_mini_cluster.cc:1428] Setting key c6ed2c1ff956dd845d799546c1da0774
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.252@KRBTEST.COM; defaulting to no policy
I20260812 05:18:10.038941 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.028868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.18.166.254:40001)
0812 05:18:10.028932 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.028933 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.029186 (+ 253us) server_negotiation.cc:407] Connection header received
0812 05:18:10.029875 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.029877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.029894 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.029945 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.030637 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.031142 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.032171 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.032395 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.032495 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.035419 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.035422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.035422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.035431 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.037223 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.037805 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.037807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.037807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.037880 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.038191 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.038192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.038192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.038324 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.038404 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.038735 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.038840 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:18:10.039160 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.028834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:40001 (local address 127.0.0.1:46844)
0812 05:18:10.029034 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.029075 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.029748 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.030057 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.030061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.030333 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.030532 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.030536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.031255 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.031256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.032064 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.032066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.032511 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.032512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.032678 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.033261 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.033282 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.035297 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.037361 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.037373 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.037376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.037715 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.037956 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.037957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.037957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.038089 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.038407 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.038416 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.038581 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.038700 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.038890 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
I20260812 05:18:10.039824 890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.252@KRBTEST.COM" created.
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:18:10.048369 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.040181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36110)
0812 05:18:10.040241 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.040254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.040300 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.040593 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.040594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.040757 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.040812 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.040814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.041809 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.041810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.042388 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.042390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.042901 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.042902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.042976 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.043516 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.043523 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.044829 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.047237 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.047238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.047239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.047425 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.047672 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.047673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.047673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.047727 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.048099 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.048099 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.048140 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.048192 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.048284 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 05:18:10.048511 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.040257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.18.166.253:36035)
0812 05:18:10.040450 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.040454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.040472 (+ 18us) server_negotiation.cc:407] Connection header received
0812 05:18:10.040490 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.040492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.040503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.040579 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.040894 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.041707 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.042482 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.042820 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.042904 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.044925 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.044935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.044936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.044945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.047121 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.047515 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.047519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.047520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.047594 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.047785 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.047786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.047786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.048023 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.048125 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.048234 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.048361 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260812 05:18:10.049230 890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:18:10.049661 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.049530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36016)
0812 05:18:10.049585 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.049638 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 05:18:10.049803 831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
I20260812 05:18:10.050271 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.050168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36030)
0812 05:18:10.050215 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.050241 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 05:18:10.050376 890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39657: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab.
Entry for principal kudu/127.18.166.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab.
I20260812 05:18:10.060714 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:39657
--webserver_interface=127.18.166.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab
--principal=kudu/127.18.166.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:40001,127.18.166.253:36035,127.18.166.252:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:10.089550 807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } attempt: 3
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:18:10.097249 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.089925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36116)
0812 05:18:10.090003 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.090022 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.090101 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.090259 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.090260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.090439 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.090508 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.090510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.091210 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.091211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.091720 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.091721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.092166 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.092167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.092258 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.092838 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.092843 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.094344 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.096075 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.096076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.096077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.096281 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.096510 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.096511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.096511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.096582 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.096870 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.096871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.096931 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.097024 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.097144 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:18:10.097249 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.090016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.18.166.253:36035)
0812 05:18:10.090093 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.090095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.090109 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:18:10.090173 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.090174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.090188 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.090243 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.090602 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.091116 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.091849 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.092079 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.092165 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.094460 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.094461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.094462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.094471 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.095973 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.096364 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.096366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.096366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.096435 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.096654 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.096655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.096656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.096783 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.096878 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.097050 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.097144 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:18:10.097952 807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:18:10.098338 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.098228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36044)
0812 05:18:10.098284 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.098317 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 05:18:10.098651 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.098558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36058)
0812 05:18:10.098605 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.098629 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 05:18:10.098745 807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39657: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
I20260812 05:18:10.099586 890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } attempt: 1
I20260812 05:18:10.099936 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.099822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36068)
0812 05:18:10.099876 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.099902 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:18:10.100278 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.100219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36078)
0812 05:18:10.100243 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.100264 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 05:18:10.100378 890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39657: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
I20260812 05:18:10.129884 807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } attempt: 1
I20260812 05:18:10.130519 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.130306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36086)
0812 05:18:10.130419 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.130466 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 05:18:10.131011 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.130919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36094)
0812 05:18:10.130956 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.130985 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 05:18:10.131108 807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39657: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
W20260812 05:18:10.148567 897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:10.148800 897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:10.148825 897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:10.150122 897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:10.150182 897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:10.150195 897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:10.150249 897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:10.150265 897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:10.151798 897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:40001,127.18.166.253:36035,127.18.166.252:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab
--principal=kudu/127.18.166.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:39657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.166.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:10.152074 897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:10.152335 897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:10.157330 903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:10.157337 904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:10.157330 906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:10.157348 897 server_base.cc:1034] running on GCE node
I20260812 05:18:10.157802 897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:10.158154 897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:10.159328 897 hybrid_clock.cc:648] HybridClock initialized: now 1786511890159317 us; error 30 us; skew 500 ppm
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:10.161903 897 init.cc:377] Logged in from keytab as kudu/127.18.166.252@KRBTEST.COM (short username kudu)
I20260812 05:18:10.162361 897 webserver.cc:460] Webserver started at http://127.18.166.252:33417/ using document root <none> and password file <none>
I20260812 05:18:10.162600 897 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:10.162671 897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:10.162845 897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:10.163944 897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data/instance:
uuid: "3ccb5456c25243548747d5d5b06dfe07"
format_stamp: "Formatted at 2026-08-12 05:18:10 on dist-test-slave-srgv"
server_key: "57db6e4d737bf9791bc13d082b6dcac3"
server_key_iv: "31fb66e6ab627bf2a0c4cc0c3dab4e7e"
server_key_version: "encryptionkey@0"
I20260812 05:18:10.164358 897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal/instance:
uuid: "3ccb5456c25243548747d5d5b06dfe07"
format_stamp: "Formatted at 2026-08-12 05:18:10 on dist-test-slave-srgv"
server_key: "57db6e4d737bf9791bc13d082b6dcac3"
server_key_iv: "31fb66e6ab627bf2a0c4cc0c3dab4e7e"
server_key_version: "encryptionkey@0"
I20260812 05:18:10.165836 897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:18:10.166592 914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:10.166772 897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:18:10.166890 897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal
uuid: "3ccb5456c25243548747d5d5b06dfe07"
format_stamp: "Formatted at 2026-08-12 05:18:10 on dist-test-slave-srgv"
server_key: "57db6e4d737bf9791bc13d082b6dcac3"
server_key_iv: "31fb66e6ab627bf2a0c4cc0c3dab4e7e"
server_key_version: "encryptionkey@0"
I20260812 05:18:10.166968 897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:10.168511 890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } attempt: 2
I20260812 05:18:10.168926 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.168794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36106)
0812 05:18:10.168858 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.168891 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:18:10.169252 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.169145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36110)
0812 05:18:10.169205 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.169230 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 05:18:10.169353 890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39657: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39657: connect: Connection refused (error 111)
I20260812 05:18:10.176517 897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:10.178660 897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:10.178843 897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:10.183187 897 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.252:39657
I20260812 05:18:10.183418 968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.252:39657 every 8 connection(s)
I20260812 05:18:10.183660 897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
I20260812 05:18:10.184602 970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:10.185902 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 897
I20260812 05:18:10.186028 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786511731160932-19099-0/minicluster-data/master-2/wal/instance
I20260812 05:18:10.186239 19099 external_mini_cluster.cc:1428] Setting key 7df144675951d35331eb17220147e0e9
I20260812 05:18:10.186383 970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:18:10.196224 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.187365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46854 (local address 127.18.166.254:40001)
0812 05:18:10.187415 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.187417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.187578 (+ 161us) server_negotiation.cc:407] Connection header received
0812 05:18:10.187727 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.187728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.187741 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.187797 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.188371 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.188856 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.189841 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.190062 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.190145 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.192577 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.192579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.192579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.192595 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.194395 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.194884 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.194885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.194885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.194950 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.195492 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.195492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.195493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.195652 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.195728 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.196015 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.196111 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:18:10.196478 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.187294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:40001 (local address 127.0.0.1:46854)
0812 05:18:10.187450 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.187500 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.187633 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.187816 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.187818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.188106 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.188273 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.188277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.188939 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.188941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.189751 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.189754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.190152 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.190154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.190286 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.190834 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.190846 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.192470 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.194482 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.194489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.194491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.194804 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.195263 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.195264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.195264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.195384 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.195725 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.195732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.195855 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.195973 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.196171 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260812 05:18:10.196661 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.186782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36124 (local address 127.18.166.252:39657)
0812 05:18:10.186981 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.186993 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.187021 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:18:10.187604 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.187611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.187636 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.187761 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.188390 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.189168 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.189998 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.190321 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.190408 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.193071 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.193079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.193079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.193085 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.194957 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.195261 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.195263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.195263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.195382 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.195610 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.195611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.195611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.195858 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.195987 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.196097 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.196240 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260812 05:18:10.197129 970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:10.204257 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.197517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.18.166.253:36035)
0812 05:18:10.197563 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.197565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.197573 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:18:10.197598 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.197599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.197609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.197666 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.197949 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.198453 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.199121 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.199339 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.199419 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.201284 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.201286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.201287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.201294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.202957 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.203373 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.203374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.203375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.203431 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.203644 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.203645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.203646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.203801 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.203892 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.204064 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.204155 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:18:10.204257 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.197442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:36035 (local address 127.0.0.1:36130)
0812 05:18:10.197482 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.197495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.197543 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.197669 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.197670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.197818 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.197863 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.197864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.198540 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.198541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.199032 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.199033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.199414 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.199415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.199490 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.199965 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.199970 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.201169 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.203041 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.203042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.203042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.203281 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.203479 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.203480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.203481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.203548 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.203884 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.203886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.203935 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.204025 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.204156 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 05:18:10.204877 970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:18:10.206015 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.197586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.18.166.254:40001)
0812 05:18:10.197621 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.197623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.197630 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:18:10.197648 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.197649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.197658 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.197713 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.198187 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.198633 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.199426 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.200050 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.200137 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.202733 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.202734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.202734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.202741 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.204395 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.204851 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.204852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.204852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.204907 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.205369 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.205370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.205370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.205561 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.205774 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.205830 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.205900 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:18:10.206357 807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } attempt: 2
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:18:10.214370 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.205218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36132)
0812 05:18:10.205254 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.205265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.205326 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.205486 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.205487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.205640 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.205677 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.205678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.206584 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.206585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.207381 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.207384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.207829 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.207830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.207894 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.209050 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.209055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.210771 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.212401 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.212402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.212403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.212629 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.212860 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.212860 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.212860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.212916 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.213949 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.213950 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.213989 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.214086 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.214256 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 05:18:10.214670 973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.205238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.18.166.252:39657)
0812 05:18:10.205284 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.205286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.205293 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:18:10.205393 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.205394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.205432 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.205480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.205753 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.206497 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.207460 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.207740 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.207810 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.210861 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.210863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.210863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.210871 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.212292 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.212714 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.212715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.212716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.212786 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.213399 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.213400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.213401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.213849 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.213939 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.214140 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.214363 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":23}
I20260812 05:18:10.215801 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.206628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36142)
0812 05:18:10.206768 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.206778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.206817 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.207035 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.207036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.207190 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.207227 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.207228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.208635 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.208637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.209222 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.209223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.209758 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.209760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.209823 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.211130 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.211136 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.212570 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.214595 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.214597 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.214597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.214842 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.215074 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.215074 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.215075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.215139 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.215457 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.215458 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.215508 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.215582 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.215693 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260812 05:18:10.215833 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.206726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.18.166.252:39657)
0812 05:18:10.206920 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.206922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.206929 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:18:10.206947 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.206948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.206959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.207032 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.207310 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.208548 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.209314 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.209666 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.209738 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.212663 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.212664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.212664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.212672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.214497 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.214926 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.214926 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.214927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.214975 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.215212 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.215213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.215213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.215359 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.215455 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.215612 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.215699 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":147,"threads_started":1}
I20260812 05:18:10.216471 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.206731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.18.166.253:36035)
0812 05:18:10.206884 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.206885 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.206892 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:18:10.206912 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.206912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.206921 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.206974 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.207572 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.208249 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.209033 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.209331 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.209390 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.212833 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.212835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.212835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.212843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.214841 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.215269 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.215271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.215271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.215336 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.215538 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.215539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.215539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.215671 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.215880 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.216300 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.216383 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260812 05:18:10.216531 970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: Bootstrap starting.
I20260812 05:18:10.217226 970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:10.217708 970 log.cc:824] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:10.218042 807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: Bootstrap starting.
I20260812 05:18:10.218456 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:18:10.218704 807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:10.218741 970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: No bootstrap required, opened a new log
I20260812 05:18:10.218915 807 log.cc:824] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:10.219715 807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: No bootstrap required, opened a new log
I20260812 05:18:10.221040 970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.221305 970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:10.221413 970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ccb5456c25243548747d5d5b06dfe07, State: Initialized, Role: FOLLOWER
I20260812 05:18:10.221674 970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.222187 807 raft_consensus.cc:348] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.222285 970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:10.222404 807 raft_consensus.cc:374] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:10.222514 807 raft_consensus.cc:729] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47eb08ffd714a6793a85acd1d579a86, State: Initialized, Role: FOLLOWER
I20260812 05:18:10.222736 807 consensus_queue.cc:260] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.223163 807 sys_catalog.cc:564] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:10.224469 897 master_runner.cc:186] Error getting master registration for 127.18.166.254:40001: OK, instance_id { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" instance_seqno: 1786511889860474 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:18:10.228025 897 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:18:10.225798 730 master_runner.cc:186] Error getting master registration for 127.18.166.253:36035: OK, instance_id { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" instance_seqno: 1786511890023551 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:18:10.228924 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.220015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46864 (local address 127.18.166.254:40001)
0812 05:18:10.220057 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.220059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.220197 (+ 138us) server_negotiation.cc:407] Connection header received
0812 05:18:10.220302 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.220302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.220314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.220344 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.221155 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.221890 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.222984 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.223280 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.223343 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.225117 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.225119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.225119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.225129 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.227006 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.227694 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.227695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.227696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.227763 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.227988 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.227988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.227989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.228133 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.228491 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.228741 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.228830 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:18:10.228976 998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:18:10.229017 998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:18:10.229229 985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.229368 985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:18:10.228937 730 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 05:18:10.230741 1009 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:18:10.230829 1009 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:18:10.233065 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.220308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.18.166.252:39657)
0812 05:18:10.220686 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.220688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.221232 (+ 544us) server_negotiation.cc:407] Connection header received
0812 05:18:10.221327 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.221327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.221339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.221368 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.223155 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.223904 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.226688 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.227006 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.227052 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.229979 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.229981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.229982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.229990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.231715 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.232154 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.232155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.232156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.232224 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.232428 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.232429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.232429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.232558 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.232669 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.232893 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.232981 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359}
I20260812 05:18:10.233182 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.220470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36152 (local address 127.18.166.253:36035)
0812 05:18:10.220653 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.220654 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.222229 (+ 1575us) server_negotiation.cc:407] Connection header received
0812 05:18:10.222328 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.222330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.222341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.222373 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.225004 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.225710 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.227467 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.227837 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.227878 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.229955 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.229957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.229957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.229965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.231862 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.232304 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.232305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.232306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.232368 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.232591 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.232592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.232592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.232726 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.232792 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.233011 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.233095 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260812 05:18:10.237576 991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.237790 991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:18:10.260519 890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } attempt: 3
Aug 12 05:18:10 dist-test-slave-srgv krb5kdc[716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:18:10.269367 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.260872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39657 (local address 127.0.0.1:36152)
0812 05:18:10.260930 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:10.260942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:10.260982 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:10.261156 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:10.261158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:10.261374 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:10.261477 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.261480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.262456 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.262457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.263277 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:10.263279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.263926 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.263927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:10.263995 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.264494 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:10.264499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:10.266181 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:10.268169 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.268170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.268171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.268418 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.268642 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:10.268643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:10.268643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:10.268714 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:10.269010 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:10.269011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:10.269063 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 05:18:10.269138 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:18:10.269247 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:18:10.269435 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:10.260959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36152 (local address 127.18.166.252:39657)
0812 05:18:10.261012 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:10.261014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:10.261024 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:18:10.261052 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:10.261053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:10.261064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:10.261135 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:10.261564 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.262366 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.263370 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:10.263675 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:10.263746 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:10.266282 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:10.266283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:10.266284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:10.266298 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:10.268066 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.268500 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.268502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.268502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.268567 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:10.268784 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:10.268785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:10.268785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:10.268935 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:10.269010 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:10.269223 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 05:18:10.269313 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:18:10.271752 890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: Bootstrap starting.
I20260812 05:18:10.272531 890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:10.272845 890 log.cc:824] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:10.273862 890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: No bootstrap required, opened a new log
I20260812 05:18:10.276010 890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.276160 890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:10.276196 890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fb9bd0196a4c38a5ceef64bb68e33c, State: Initialized, Role: FOLLOWER
I20260812 05:18:10.276376 890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.276744 890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:10.276716 1014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.276942 1014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:18:10.278740 1027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:18:10.278861 1027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:18:10.281389 815 master_runner.cc:263] No leader master found from master 56fb9bd0196a4c38a5ceef64bb68e33c
I20260812 05:18:10.281502 815 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 05:18:10.358347 754 tablet.cc:2369] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:18:10.429590 920 tablet.cc:2369] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:10.432312 991 raft_consensus.cc:482] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:10.432447 991 raft_consensus.cc:504] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.432767 991 leader_election.cc:288] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56fb9bd0196a4c38a5ceef64bb68e33c (127.18.166.253:36035), 3ccb5456c25243548747d5d5b06dfe07 (127.18.166.252:39657)
I20260812 05:18:10.433104 942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f47eb08ffd714a6793a85acd1d579a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ccb5456c25243548747d5d5b06dfe07" is_pre_election: true
I20260812 05:18:10.433136 858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f47eb08ffd714a6793a85acd1d579a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" is_pre_election: true
I20260812 05:18:10.433288 942 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f47eb08ffd714a6793a85acd1d579a86 in term 0.
I20260812 05:18:10.433295 858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f47eb08ffd714a6793a85acd1d579a86 in term 0.
I20260812 05:18:10.433537 751 leader_election.cc:302] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fb9bd0196a4c38a5ceef64bb68e33c, f47eb08ffd714a6793a85acd1d579a86; no voters:
I20260812 05:18:10.433691 991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:18:10.433756 991 raft_consensus.cc:482] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:18:10.433776 991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:10.434662 991 raft_consensus.cc:504] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.434850 991 leader_election.cc:288] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [CANDIDATE]: Term 1 election: Requested vote from peers 56fb9bd0196a4c38a5ceef64bb68e33c (127.18.166.253:36035), 3ccb5456c25243548747d5d5b06dfe07 (127.18.166.252:39657)
I20260812 05:18:10.435087 858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f47eb08ffd714a6793a85acd1d579a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c"
I20260812 05:18:10.435094 942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f47eb08ffd714a6793a85acd1d579a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ccb5456c25243548747d5d5b06dfe07"
I20260812 05:18:10.435190 858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:10.435213 942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:10.436177 942 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f47eb08ffd714a6793a85acd1d579a86 in term 1.
I20260812 05:18:10.436393 747 leader_election.cc:302] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ccb5456c25243548747d5d5b06dfe07, f47eb08ffd714a6793a85acd1d579a86; no voters:
I20260812 05:18:10.436492 858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f47eb08ffd714a6793a85acd1d579a86 in term 1.
I20260812 05:18:10.436504 991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:10.436642 991 raft_consensus.cc:686] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [term 1 LEADER]: Becoming Leader. State: Replica: f47eb08ffd714a6793a85acd1d579a86, State: Running, Role: LEADER
I20260812 05:18:10.436770 991 consensus_queue.cc:237] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } }
I20260812 05:18:10.437208 991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47eb08ffd714a6793a85acd1d579a86. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.437289 991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:10.437570 1031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:10.437768 1031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:10.439174 942 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [term 1 FOLLOWER]: Refusing update from remote peer f47eb08ffd714a6793a85acd1d579a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:10.439201 858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [term 1 FOLLOWER]: Refusing update from remote peer f47eb08ffd714a6793a85acd1d579a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:10.439577 1029 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:10.439723 991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:10.440956 985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47eb08ffd714a6793a85acd1d579a86. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.440964 1014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47eb08ffd714a6793a85acd1d579a86. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.441056 985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:18:10.441074 1014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:18:10.442202 991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.442296 991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:10.442384 985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.442399 991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.442464 985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:18:10.442464 991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:10.442668 1034 mvcc.cc:205] Tried to move back new op lower bound from 7317552703236550656 to 7317552703229833216. Current Snapshot: MvccSnapshot[applied={T|T < 7317552703236550656}]
I20260812 05:18:10.442764 1014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f47eb08ffd714a6793a85acd1d579a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f47eb08ffd714a6793a85acd1d579a86" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 40001 } } peers { permanent_uuid: "56fb9bd0196a4c38a5ceef64bb68e33c" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 36035 } } peers { permanent_uuid: "3ccb5456c25243548747d5d5b06dfe07" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39657 } } }
I20260812 05:18:10.442848 1014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:18:10.443256 1033 mvcc.cc:205] Tried to move back new op lower bound from 7317552703236550656 to 7317552703229833216. Current Snapshot: MvccSnapshot[applied={T|T < 7317552703236550656 or (T in {7317552703236550656})}]
I20260812 05:18:10.443691 1031 catalog_manager.cc:1347] Generated new cluster ID: 096ab8b2ee8b4fa09f3555030345ad6d
I20260812 05:18:10.443744 1031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:10.474283 1031 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:18:10.475061 1031 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:10.486094 1031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f47eb08ffd714a6793a85acd1d579a86: Generated new TSK 0
I20260812 05:18:10.486380 1031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:18:11.229534 998 catalog_manager.cc:1259] Loaded cluster ID: 096ab8b2ee8b4fa09f3555030345ad6d
I20260812 05:18:11.229696 998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: loading cluster ID for follower catalog manager: success
I20260812 05:18:11.230583 998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: acquiring CA information for follower catalog manager: success
I20260812 05:18:11.230914 998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ccb5456c25243548747d5d5b06dfe07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:18:11.279366 1027 catalog_manager.cc:1259] Loaded cluster ID: 096ab8b2ee8b4fa09f3555030345ad6d
I20260812 05:18:11.279453 1027 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: loading cluster ID for follower catalog manager: success
I20260812 05:18:11.280324 1027 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: acquiring CA information for follower catalog manager: success
I20260812 05:18:11.280668 1027 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56fb9bd0196a4c38a5ceef64bb68e33c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:18:11.547817 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 730
I20260812 05:18:11.554414 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 815
I20260812 05:18:11.560766 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 897
2026-08-12T05:18:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3030 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:18:11 dist-test-slave-srgv krb5kdc[1046](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:18:11 dist-test-slave-srgv krb5kdc[1046](info): set up 2 sockets
Aug 12 05:18:11 dist-test-slave-srgv krb5kdc[1046](info): commencing operation
krb5kdc: starting...
W20260812 05:18:12.640784 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:18:12 dist-test-slave-srgv krb5kdc[1046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:18:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:18:12.772038 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42363
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:42363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:12.854962 1060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:12.855176 1060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:12.855201 1060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:12.856285 1060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:12.856324 1060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:12.856338 1060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:12.856351 1060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:12.856364 1060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:12.857836 1060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:42363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:12.858038 1060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:12.858227 1060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:12.863011 1068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:12.863011 1065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:12.863111 1066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:12.863358 1060 server_base.cc:1034] running on GCE node
I20260812 05:18:12.863620 1060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:12.863910 1060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:12.865075 1060 hybrid_clock.cc:648] HybridClock initialized: now 1786511892865065 us; error 30 us; skew 500 ppm
Aug 12 05:18:12 dist-test-slave-srgv krb5kdc[1046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511892, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:12.867823 1060 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:12.868278 1060 webserver.cc:460] Webserver started at http://127.18.166.254:40767/ using document root <none> and password file <none>
I20260812 05:18:12.868521 1060 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:12.868589 1060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:12.868743 1060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:12.869907 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "06a17495f3314ab4a97187a65f3d5ac3"
format_stamp: "Formatted at 2026-08-12 05:18:12 on dist-test-slave-srgv"
server_key: "349c79f3ce7925c2955d210480ffee1b"
server_key_iv: "9b06ae380ef5c44b4ea9439f5394a2d0"
server_key_version: "encryptionkey@0"
I20260812 05:18:12.870281 1060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "06a17495f3314ab4a97187a65f3d5ac3"
format_stamp: "Formatted at 2026-08-12 05:18:12 on dist-test-slave-srgv"
server_key: "349c79f3ce7925c2955d210480ffee1b"
server_key_iv: "9b06ae380ef5c44b4ea9439f5394a2d0"
server_key_version: "encryptionkey@0"
I20260812 05:18:12.871647 1060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:18:12.872423 1075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:12.872644 1060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:12.872753 1060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "06a17495f3314ab4a97187a65f3d5ac3"
format_stamp: "Formatted at 2026-08-12 05:18:12 on dist-test-slave-srgv"
server_key: "349c79f3ce7925c2955d210480ffee1b"
server_key_iv: "9b06ae380ef5c44b4ea9439f5394a2d0"
server_key_version: "encryptionkey@0"
I20260812 05:18:12.872828 1060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:12.882134 1060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:12.884459 1060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:12.884621 1060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:12.888767 1060 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:42363
I20260812 05:18:12.888800 1130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:42363 every 8 connection(s)
I20260812 05:18:12.889174 1060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:12.890158 1131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:12.892422 1131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3: Bootstrap starting.
I20260812 05:18:12.893013 1131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:12.893319 1131 log.cc:824] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:12.894069 1131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3: No bootstrap required, opened a new log
I20260812 05:18:12.895756 1131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a17495f3314ab4a97187a65f3d5ac3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42363 } }
I20260812 05:18:12.895889 1131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:12.895911 1131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06a17495f3314ab4a97187a65f3d5ac3, State: Initialized, Role: FOLLOWER
I20260812 05:18:12.896080 1131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a17495f3314ab4a97187a65f3d5ac3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42363 } }
I20260812 05:18:12.896178 1131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:12.896229 1131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:12.896278 1131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:12.896795 1131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a17495f3314ab4a97187a65f3d5ac3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42363 } }
I20260812 05:18:12.896906 1131 leader_election.cc:302] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06a17495f3314ab4a97187a65f3d5ac3; no voters:
I20260812 05:18:12.897126 1131 leader_election.cc:288] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:18:12.897152 1135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:12.897305 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1060
I20260812 05:18:12.897385 1135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 06a17495f3314ab4a97187a65f3d5ac3, State: Running, Role: LEADER
I20260812 05:18:12.897431 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:18:12.897430 1131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:12.897486 1135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a17495f3314ab4a97187a65f3d5ac3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42363 } }
I20260812 05:18:12.897687 19099 external_mini_cluster.cc:1428] Setting key 1eb653d9e4530fe8bf770b2eaad5c431
I20260812 05:18:12.897851 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06a17495f3314ab4a97187a65f3d5ac3. Latest consensus state: current_term: 1 leader_uuid: "06a17495f3314ab4a97187a65f3d5ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a17495f3314ab4a97187a65f3d5ac3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42363 } } }
I20260812 05:18:12.897948 1136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06a17495f3314ab4a97187a65f3d5ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a17495f3314ab4a97187a65f3d5ac3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42363 } } }
I20260812 05:18:12.897990 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:12.898003 1136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:12.898559 1144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:12.899084 1144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:12.900692 1144 catalog_manager.cc:1347] Generated new cluster ID: dc18f44cbab4446da06c2ffdf7e7d4bb
I20260812 05:18:12.900746 1144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:18:12 dist-test-slave-srgv krb5kdc[1046](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:12.908746 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:12.898357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32874 (local address 127.18.166.254:42363)
0812 05:18:12.898916 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:12.898926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:12.898957 (+ 31us) server_negotiation.cc:407] Connection header received
0812 05:18:12.899072 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:12.899078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:12.899147 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:12.900100 (+ 953us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:12.900212 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:12.901648 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:12.902526 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:12.902965 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:12.903093 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:12.905456 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:12.905459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:12.905460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:12.905474 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:12.907166 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:12.907539 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:12.907542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:12.907543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:12.907646 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:12.907870 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:12.907871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:12.907872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:12.908107 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:12.908287 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:12.908386 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 05:18:12.908535 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":84,"threads_started":1}
I20260812 05:18:12.909840 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:18:12.913590 1144 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:18:12.914113 1144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:12.918959 1144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06a17495f3314ab4a97187a65f3d5ac3: Generated new TSK 0
I20260812 05:18:12.919139 1144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:18:12.963811 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1060
2026-08-12T05:18:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1401 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:18:13 dist-test-slave-srgv krb5kdc[1160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:18:13 dist-test-slave-srgv krb5kdc[1160](info): set up 2 sockets
Aug 12 05:18:13 dist-test-slave-srgv krb5kdc[1160](info): commencing operation
krb5kdc: starting...
W20260812 05:18:14.039949 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:18:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:18:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:18:14.172397 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34219
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:34219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:14.256366 1175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:14.256605 1175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:14.256630 1175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:14.257965 1175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:14.258014 1175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:14.258029 1175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:14.258044 1175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:14.258056 1175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:14.259577 1175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:34219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:14.259794 1175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:14.259972 1175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:14.264971 1180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.264981 1183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.264979 1181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:14.265362 1175 server_base.cc:1034] running on GCE node
I20260812 05:18:14.265640 1175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:14.265911 1175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:14.267063 1175 hybrid_clock.cc:648] HybridClock initialized: now 1786511894267051 us; error 34 us; skew 500 ppm
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:14.269711 1175 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:14.270196 1175 webserver.cc:460] Webserver started at http://127.18.166.254:37109/ using document root <none> and password file <none>
I20260812 05:18:14.270404 1175 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:14.270449 1175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:14.270542 1175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:14.271571 1175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "873f064675b74400a3d30e8354cb6fe3"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c9b5576fb2d6eab66f0c803ca5eb96d0"
server_key_iv: "c53adf06664a4cd7c5fb01817f895696"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.271894 1175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "873f064675b74400a3d30e8354cb6fe3"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c9b5576fb2d6eab66f0c803ca5eb96d0"
server_key_iv: "c53adf06664a4cd7c5fb01817f895696"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.273269 1175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:18:14.274132 1190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.274348 1175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:14.274446 1175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "873f064675b74400a3d30e8354cb6fe3"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c9b5576fb2d6eab66f0c803ca5eb96d0"
server_key_iv: "c53adf06664a4cd7c5fb01817f895696"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.274506 1175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:14.287041 1175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:14.289173 1175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:14.289294 1175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:14.293262 1175 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:34219
I20260812 05:18:14.293339 1246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:34219 every 8 connection(s)
I20260812 05:18:14.293715 1175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:14.294725 1247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:14.296965 1247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap starting.
I20260812 05:18:14.297600 1247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:14.297791 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1175
I20260812 05:18:14.297904 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:18:14.297935 1247 log.cc:824] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:14.298120 19099 external_mini_cluster.cc:1428] Setting key e39f7d4598fcc09c4526aa168fc1bcfa
I20260812 05:18:14.298857 1247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: No bootstrap required, opened a new log
I20260812 05:18:14.301587 1247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:14.301756 1247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:14.301800 1247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873f064675b74400a3d30e8354cb6fe3, State: Initialized, Role: FOLLOWER
I20260812 05:18:14.301931 1247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:14.302026 1247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:14.302049 1247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:14.302124 1247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:14.302727 1247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:14.302829 1247 leader_election.cc:302] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873f064675b74400a3d30e8354cb6fe3; no voters:
I20260812 05:18:14.302980 1247 leader_election.cc:288] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:18:14.303045 1252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:14.303211 1247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:14.303232 1252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 LEADER]: Becoming Leader. State: Replica: 873f064675b74400a3d30e8354cb6fe3, State: Running, Role: LEADER
I20260812 05:18:14.304736 1252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:14.305385 1254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873f064675b74400a3d30e8354cb6fe3. Latest consensus state: current_term: 1 leader_uuid: "873f064675b74400a3d30e8354cb6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } } }
I20260812 05:18:14.305388 1253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "873f064675b74400a3d30e8354cb6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } } }
I20260812 05:18:14.305514 1254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:14.305521 1253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:14.305809 1266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:14.306439 1266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:14.309233 1266 catalog_manager.cc:1347] Generated new cluster ID: d67e26e8b28541059aa50ff0ef81e4b4
I20260812 05:18:14.309288 1266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:14.310415 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.298703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.18.166.254:34219)
0812 05:18:14.298905 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.298914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.298950 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:18:14.299451 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.299459 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.299523 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.299676 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:14.300287 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.301728 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.302672 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.303308 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.303431 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.306500 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:14.306504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:14.306504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:14.306516 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:14.308384 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.308850 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.308853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.308854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.308953 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.309171 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.309172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.309172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.309511 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:14.309654 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.310067 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.310200 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260812 05:18:14.329293 1266 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:18:14.329914 1266 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:14.334753 1266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Generated new TSK 0
I20260812 05:18:14.334942 1266 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:18:14.372051 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34219
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:14.455796 1274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:14.456024 1274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:14.456049 1274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:14.457259 1274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:14.457311 1274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:14.457340 1274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:14.458894 1274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:14.459159 1274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:14.459429 1274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:14.464717 1281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.464725 1283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.464783 1280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:14.464990 1274 server_base.cc:1034] running on GCE node
I20260812 05:18:14.465217 1274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:14.465554 1274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:14.466717 1274 hybrid_clock.cc:648] HybridClock initialized: now 1786511894466701 us; error 32 us; skew 500 ppm
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:14.469650 1274 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:14.470074 1274 webserver.cc:460] Webserver started at http://127.18.166.193:35681/ using document root <none> and password file <none>
I20260812 05:18:14.470263 1274 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:14.470304 1274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:14.470394 1274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:14.471525 1274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "7e95b719833149158e6b28ef0ba5955a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "ee788ac64504f87845bed5c31d74c3cd"
server_key_iv: "f4f1368e024412b239122728b68a1238"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.471868 1274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "7e95b719833149158e6b28ef0ba5955a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "ee788ac64504f87845bed5c31d74c3cd"
server_key_iv: "f4f1368e024412b239122728b68a1238"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.473193 1274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:18:14.474045 1290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.474220 1274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:18:14.474311 1274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "7e95b719833149158e6b28ef0ba5955a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "ee788ac64504f87845bed5c31d74c3cd"
server_key_iv: "f4f1368e024412b239122728b68a1238"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.474370 1274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:14.497814 1274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:14.500216 1274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:14.500341 1274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:14.500548 1274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:14.500891 1274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:14.500928 1274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.500952 1274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:14.500998 1274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.507791 1274 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:44397
I20260812 05:18:14.507853 1409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:44397 every 8 connection(s)
I20260812 05:18:14.508253 1274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:14.517956 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.509145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:41549 (local address 127.18.166.254:34219)
0812 05:18:14.509197 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.509200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.509512 (+ 312us) server_negotiation.cc:407] Connection header received
0812 05:18:14.510004 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.510005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.510016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.510086 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:14.511014 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.511529 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.512493 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.512701 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.512762 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.515042 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:14.515044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:14.515044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:14.515051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:14.516535 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.516918 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.516919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.516919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.516971 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.517186 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.517186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.517186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.517292 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:14.517345 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.517725 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.517805 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:18:14.518030 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1274
I20260812 05:18:14.518033 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.509134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:41549)
0812 05:18:14.509353 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:14.509396 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:14.509916 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:14.510141 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:14.510144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:14.510728 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:14.510922 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.510928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.511616 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.511616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.512404 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.512406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.512787 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.512788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.512896 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.513536 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:14.513544 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:14.514952 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:14.516617 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:14.516623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:14.516625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:14.516849 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:14.517031 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:14.517032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:14.517032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:14.517131 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:14.517389 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:14.517396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:14.517596 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 05:18:14.517700 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 05:18:14.517833 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
I20260812 05:18:14.518134 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:18:14.518380 19099 external_mini_cluster.cc:1428] Setting key c452a0ec6f2ed2526f94ffe9375ee9e7
I20260812 05:18:14.518599 1411 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34219
I20260812 05:18:14.518682 1411 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:14.518901 1411 heartbeater.cc:511] Master 127.18.166.254:34219 requested a full tablet report, sending...
I20260812 05:18:14.519456 1208 ts_manager.cc:194] Registered new tserver with Master: 7e95b719833149158e6b28ef0ba5955a (127.18.166.193:44397)
I20260812 05:18:14.520109 1208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:41549
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:18:14.549685 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34219
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:14.632272 1416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:14.632490 1416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:14.632514 1416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:14.633847 1416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:14.633898 1416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:14.633924 1416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:14.635456 1416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:14.635732 1416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:14.635977 1416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:18:14.641383 1416 server_base.cc:1034] running on GCE node
W20260812 05:18:14.641350 1424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.641485 1421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.641347 1422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:14.641808 1416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:14.642019 1416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:14.643165 1416 hybrid_clock.cc:648] HybridClock initialized: now 1786511894643152 us; error 35 us; skew 500 ppm
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:14.645965 1416 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:14.646456 1416 webserver.cc:460] Webserver started at http://127.18.166.194:36653/ using document root <none> and password file <none>
I20260812 05:18:14.646700 1416 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:14.646770 1416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:14.646934 1416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:14.648027 1416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "02e07d591a8542f49b95ff3aef77eca7"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "05b653673d59b0ab8f46a37ad332978c"
server_key_iv: "d9a461655738ae1ba1f02086ef682cdd"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.648398 1416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "02e07d591a8542f49b95ff3aef77eca7"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "05b653673d59b0ab8f46a37ad332978c"
server_key_iv: "d9a461655738ae1ba1f02086ef682cdd"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.649842 1416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:18:14.650602 1431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.650775 1416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:18:14.650876 1416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "02e07d591a8542f49b95ff3aef77eca7"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "05b653673d59b0ab8f46a37ad332978c"
server_key_iv: "d9a461655738ae1ba1f02086ef682cdd"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.650959 1416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:14.662878 1416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:14.665390 1416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:14.665699 1416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:14.665957 1416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:14.666283 1416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:14.666339 1416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.666386 1416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:14.666415 1416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.672545 1416 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38797
I20260812 05:18:14.672627 1552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38797 every 8 connection(s)
I20260812 05:18:14.672958 1416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:18:14.675006 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1416
I20260812 05:18:14.675098 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:18:14.675288 19099 external_mini_cluster.cc:1428] Setting key 2f9c794d17739a81a56c8950f918bda6
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:14.683704 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.673790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:44005 (local address 127.18.166.254:34219)
0812 05:18:14.673869 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.673872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.674025 (+ 153us) server_negotiation.cc:407] Connection header received
0812 05:18:14.674489 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.674490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.674500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.674552 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:14.675697 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.676383 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.677587 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.677928 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.678028 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.680660 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:14.680661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:14.680661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:14.680670 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:14.682250 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.682697 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.682699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.682699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.682760 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.683029 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.683030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.683031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.683150 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:14.683232 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.683519 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.683608 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:18:14.683925 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.673731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:44005)
0812 05:18:14.673902 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:14.673939 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:14.674404 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:14.674611 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:14.674614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:14.675280 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:14.675603 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.675610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.676481 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.676481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.677506 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.677509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.678042 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.678043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.678227 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.678863 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:14.678874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:14.680555 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:14.682357 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:14.682365 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:14.682367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:14.682601 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:14.682830 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:14.682830 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:14.682831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:14.682952 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:14.683225 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:14.683233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:14.683381 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 05:18:14.683487 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 05:18:14.683657 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:18:14.684520 1553 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34219
I20260812 05:18:14.684620 1553 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:14.684829 1553 heartbeater.cc:511] Master 127.18.166.254:34219 requested a full tablet report, sending...
I20260812 05:18:14.685397 1208 ts_manager.cc:194] Registered new tserver with Master: 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797)
I20260812 05:18:14.685840 1208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:44005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:18:14.704520 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34219
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:14.788682 1558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:14.788908 1558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:14.788932 1558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:14.790197 1558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:14.790241 1558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:14.790268 1558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:14.791754 1558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:14.792013 1558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:14.792259 1558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:14.797489 1564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.797489 1568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:14.797564 1565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:14.797622 1558 server_base.cc:1034] running on GCE node
I20260812 05:18:14.797912 1558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:14.798156 1558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:14.799330 1558 hybrid_clock.cc:648] HybridClock initialized: now 1786511894799316 us; error 30 us; skew 500 ppm
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:14.801968 1558 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:14.802465 1558 webserver.cc:460] Webserver started at http://127.18.166.195:42415/ using document root <none> and password file <none>
I20260812 05:18:14.802701 1558 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:14.802770 1558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:14.802946 1558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:14.804247 1558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "8729388ea95b4467a1881dd8d119cb1a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c640919f257d135a709700ea0c854899"
server_key_iv: "8559d488a5174211c65bfe3fce177590"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.804683 1558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "8729388ea95b4467a1881dd8d119cb1a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c640919f257d135a709700ea0c854899"
server_key_iv: "8559d488a5174211c65bfe3fce177590"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.806327 1558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 05:18:14.807165 1576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.807382 1558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:14.807487 1558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "8729388ea95b4467a1881dd8d119cb1a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c640919f257d135a709700ea0c854899"
server_key_iv: "8559d488a5174211c65bfe3fce177590"
server_key_version: "encryptionkey@0"
I20260812 05:18:14.807571 1558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:14.829587 1558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:14.831602 1558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:14.831776 1558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:14.832029 1558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:14.832348 1558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:14.832405 1558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.832453 1558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:14.832477 1558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:14.839324 1558 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:33969
I20260812 05:18:14.839395 1699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:33969 every 8 connection(s)
I20260812 05:18:14.839795 1558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:18:14 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511894, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:14.848932 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.840582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:45003 (local address 127.18.166.254:34219)
0812 05:18:14.840663 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.840667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.840908 (+ 241us) server_negotiation.cc:407] Connection header received
0812 05:18:14.841481 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.841482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.841496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.841568 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:14.842527 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.843029 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.844020 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.844223 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.844290 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.846491 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:14.846493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:14.846493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:14.846500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:14.847751 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.848128 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.848129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.848129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.848180 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.848390 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.848391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.848391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.848496 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:14.848548 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.848798 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.848863 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:18:14.849174 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.840571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:45003)
0812 05:18:14.840752 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:14.840790 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:14.841333 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:14.841620 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:14.841623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:14.842211 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:14.842405 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.842409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.843136 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.843137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.843927 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.843929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.844314 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.844315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.844415 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.844991 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:14.844999 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:14.846401 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:14.847829 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:14.847835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:14.847837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:14.848054 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:14.848238 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:14.848238 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:14.848238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:14.848335 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:14.848555 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:14.848560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:14.848684 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 05:18:14.848768 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:18:14.848891 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260812 05:18:14.849768 1700 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34219
I20260812 05:18:14.849907 1700 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:14.850071 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1558
I20260812 05:18:14.850111 1700 heartbeater.cc:511] Master 127.18.166.254:34219 requested a full tablet report, sending...
I20260812 05:18:14.850147 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:18:14.850366 19099 external_mini_cluster.cc:1428] Setting key ec6abbb50f5739705abd2ac026af62b3
I20260812 05:18:14.850602 1208 ts_manager.cc:194] Registered new tserver with Master: 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
I20260812 05:18:14.851068 1208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:45003
I20260812 05:18:14.851261 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:18:14.858475 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.852520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.18.166.254:34219)
0812 05:18:14.852582 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.852584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.852704 (+ 120us) server_negotiation.cc:407] Connection header received
0812 05:18:14.852765 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.852766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.852775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.852822 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:14.853392 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.853962 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.854653 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.854874 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.854927 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.856029 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:14.856030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:14.856030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:14.856036 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:14.857507 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.857857 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.857857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.857858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.857911 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:14.858048 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:14.858049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:14.858050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:14.858183 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:14.858258 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.858357 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.858413 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:18:14.860162 1208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:18:14.861891 1208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:18:14.868896 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.864541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.18.166.193:44397)
0812 05:18:14.864765 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.864771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.865242 (+ 471us) server_negotiation.cc:407] Connection header received
0812 05:18:14.865279 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.865281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.865293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.865374 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:14.865531 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.866209 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.867752 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.868506 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.868606 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.868669 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.868771 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.868810 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":107,"threads_started":1}
I20260812 05:18:14.868899 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.864214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33969 (local address 127.0.0.1:59010)
0812 05:18:14.864320 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:14.864360 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:14.864422 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:14.865003 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:14.865005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:14.865008 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:14.865079 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.865081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.866561 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.866562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.867632 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.867635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.868606 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.868608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.868689 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.868693 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:14.868733 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 05:18:14.868799 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":63,"threads_started":1}
I20260812 05:18:14.868902 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.864427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44397 (local address 127.0.0.1:36882)
0812 05:18:14.865112 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:14.865122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:14.865176 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:14.865372 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:14.865373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:14.865375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:14.865472 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.865474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.866310 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.866311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.867634 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.867635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.868588 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.868589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.868671 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.868686 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:18:14.868759 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 05:18:14.868807 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":97,"threads_started":1}
I20260812 05:18:14.869438 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.864386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.18.166.195:33969)
0812 05:18:14.864832 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.864835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.864845 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:18:14.864875 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.864876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.864888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.864997 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:14.865288 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.866440 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.867751 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.868508 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.869206 (+ 698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.869271 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.869295 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.869337 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":118,"threads_started":1}
I20260812 05:18:14.869585 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.864232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38797 (local address 127.0.0.1:60464)
0812 05:18:14.864556 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:14.864564 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:14.864602 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:14.865168 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:14.865169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:14.865171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:14.865205 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.865206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.867739 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.867740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.868428 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:14.868429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.869357 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.869358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:14.869420 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.869447 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 05:18:14.869481 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 05:18:14.869519 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":83,"threads_started":1}
I20260812 05:18:14.870005 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:14.864625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60464 (local address 127.18.166.194:38797)
0812 05:18:14.864799 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:14.864803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:14.864812 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:18:14.864890 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:14.864892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:14.864908 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:14.864986 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:14.865258 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.866437 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.868527 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:14.869263 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:14.869842 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:14.869889 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:14.869912 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 05:18:14.869944 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260812 05:18:14.870085 1342 tablet_service.cc:1467] Processing CreateTablet for tablet 814d353ecda94d61a5cec951b7415e7e (DEFAULT_TABLE table=test-table [id=1331e72f910c4a43b22af8067fa9f214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:14.870517 1342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 814d353ecda94d61a5cec951b7415e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:14.870723 1483 tablet_service.cc:1467] Processing CreateTablet for tablet 814d353ecda94d61a5cec951b7415e7e (DEFAULT_TABLE table=test-table [id=1331e72f910c4a43b22af8067fa9f214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:14.871037 1483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 814d353ecda94d61a5cec951b7415e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:14.873816 1630 tablet_service.cc:1467] Processing CreateTablet for tablet 814d353ecda94d61a5cec951b7415e7e (DEFAULT_TABLE table=test-table [id=1331e72f910c4a43b22af8067fa9f214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:14.874128 1630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 814d353ecda94d61a5cec951b7415e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:14.874121 1718 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Bootstrap starting.
I20260812 05:18:14.874416 1719 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Bootstrap starting.
I20260812 05:18:14.874897 1719 tablet_bootstrap.cc:654] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:14.874941 1718 tablet_bootstrap.cc:654] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:14.875322 1719 log.cc:824] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:14.875527 1718 log.cc:824] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:14.876004 1719 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: No bootstrap required, opened a new log
I20260812 05:18:14.876049 1719 ts_tablet_manager.cc:1397] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:18:14.877235 1722 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Bootstrap starting.
I20260812 05:18:14.878010 1722 tablet_bootstrap.cc:654] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:14.877909 1719 raft_consensus.cc:348] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:14.878072 1719 raft_consensus.cc:374] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:14.878105 1718 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: No bootstrap required, opened a new log
I20260812 05:18:14.878121 1719 raft_consensus.cc:729] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e07d591a8542f49b95ff3aef77eca7, State: Initialized, Role: FOLLOWER
I20260812 05:18:14.878165 1718 ts_tablet_manager.cc:1397] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:18:14.878252 1719 consensus_queue.cc:260] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:14.878564 1719 ts_tablet_manager.cc:1428] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:18:14.878612 1722 log.cc:824] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:14.878710 1553 heartbeater.cc:503] Master 127.18.166.254:34219 was elected leader, sending a full tablet report...
I20260812 05:18:14.879384 1722 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: No bootstrap required, opened a new log
I20260812 05:18:14.879463 1722 ts_tablet_manager.cc:1397] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:14.880127 1718 raft_consensus.cc:348] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:14.880285 1718 raft_consensus.cc:374] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:14.880334 1718 raft_consensus.cc:729] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e95b719833149158e6b28ef0ba5955a, State: Initialized, Role: FOLLOWER
I20260812 05:18:14.880445 1718 consensus_queue.cc:260] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:14.880733 1718 ts_tablet_manager.cc:1428] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:18:14.880775 1411 heartbeater.cc:503] Master 127.18.166.254:34219 was elected leader, sending a full tablet report...
I20260812 05:18:14.881392 1722 raft_consensus.cc:348] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:14.881582 1722 raft_consensus.cc:374] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:14.881651 1722 raft_consensus.cc:729] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8729388ea95b4467a1881dd8d119cb1a, State: Initialized, Role: FOLLOWER
I20260812 05:18:14.881767 1722 consensus_queue.cc:260] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:14.882043 1722 ts_tablet_manager.cc:1428] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:18:14.882045 1700 heartbeater.cc:503] Master 127.18.166.254:34219 was elected leader, sending a full tablet report...
W20260812 05:18:14.923591 1554 tablet.cc:2369] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:18:15.009825 1412 tablet.cc:2369] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:15.065234 1725 raft_consensus.cc:482] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:15.065369 1725 raft_consensus.cc:504] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.065783 1725 leader_election.cc:288] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
I20260812 05:18:15.070149 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.066095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:56301 (local address 127.18.166.194:38797)
0812 05:18:15.066152 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.066155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.066265 (+ 110us) server_negotiation.cc:407] Connection header received
0812 05:18:15.066330 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.066331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.066343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.066418 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:15.066703 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.067486 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.069072 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.069704 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.069793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.069802 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.070029 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.070076 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:18:15.070149 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.066103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:59839 (local address 127.18.166.195:33969)
0812 05:18:15.066152 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.066155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.066264 (+ 109us) server_negotiation.cc:407] Connection header received
0812 05:18:15.066323 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.066324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.066334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.066418 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:15.066704 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.067468 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.069065 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.069704 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.069793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.069802 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.070029 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.070084 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:18:15.070158 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.066008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38797 (local address 127.18.166.193:56301)
0812 05:18:15.066174 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.066186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.066257 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.066444 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.066445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.066447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:15.066579 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.066581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.067571 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.067572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.068940 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.068943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.069806 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.069807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.069897 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.069904 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.069968 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.070034 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":114,"thread_start_us":73,"threads_started":1}
I20260812 05:18:15.070158 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.066007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33969 (local address 127.18.166.193:59839)
0812 05:18:15.066173 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.066186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.066257 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.066444 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.066445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.066448 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:15.066580 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.066581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.067572 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.067572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.068940 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.068943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.069807 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.069808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.069897 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.069905 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.069968 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.070034 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":105,"spinlock_wait_cycles":13568,"thread_start_us":107,"threads_started":1}
I20260812 05:18:15.070578 1505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "814d353ecda94d61a5cec951b7415e7e" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02e07d591a8542f49b95ff3aef77eca7" is_pre_election: true
I20260812 05:18:15.070768 1505 raft_consensus.cc:2454] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 0.
I20260812 05:18:15.070850 1652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "814d353ecda94d61a5cec951b7415e7e" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8729388ea95b4467a1881dd8d119cb1a" is_pre_election: true
I20260812 05:18:15.071013 1652 raft_consensus.cc:2454] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 0.
I20260812 05:18:15.071038 1291 leader_election.cc:302] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02e07d591a8542f49b95ff3aef77eca7, 7e95b719833149158e6b28ef0ba5955a; no voters:
I20260812 05:18:15.071197 1725 raft_consensus.cc:2781] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:18:15.071276 1725 raft_consensus.cc:482] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:18:15.071321 1725 raft_consensus.cc:3037] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:15.072059 1725 raft_consensus.cc:504] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.072211 1725 leader_election.cc:288] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 election: Requested vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
I20260812 05:18:15.072403 1652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "814d353ecda94d61a5cec951b7415e7e" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8729388ea95b4467a1881dd8d119cb1a"
I20260812 05:18:15.072403 1505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "814d353ecda94d61a5cec951b7415e7e" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02e07d591a8542f49b95ff3aef77eca7"
I20260812 05:18:15.072481 1652 raft_consensus.cc:3037] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:15.072481 1505 raft_consensus.cc:3037] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:15.073362 1505 raft_consensus.cc:2454] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 1.
I20260812 05:18:15.073362 1652 raft_consensus.cc:2454] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 1.
I20260812 05:18:15.073561 1292 leader_election.cc:302] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e95b719833149158e6b28ef0ba5955a, 8729388ea95b4467a1881dd8d119cb1a; no voters:
I20260812 05:18:15.073679 1725 raft_consensus.cc:2781] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:15.073863 1725 raft_consensus.cc:686] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 LEADER]: Becoming Leader. State: Replica: 7e95b719833149158e6b28ef0ba5955a, State: Running, Role: LEADER
I20260812 05:18:15.073962 1725 consensus_queue.cc:237] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.074805 1208 catalog_manager.cc:5495] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e95b719833149158e6b28ef0ba5955a (127.18.166.193). New cstate: current_term: 1 leader_uuid: "7e95b719833149158e6b28ef0ba5955a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } health_report { overall_health: HEALTHY } } }
W20260812 05:18:15.090479 1702 tablet.cc:2369] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:15.094725 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.092418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.18.166.193:44397)
0812 05:18:15.092475 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.092477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.092493 (+ 16us) server_negotiation.cc:407] Connection header received
0812 05:18:15.092530 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.092531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.092549 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.092612 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:15.092753 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.093217 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.094042 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.094232 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.094280 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.094459 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:15.094555 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:18:15.094615 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.094649 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.094666 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 05:18:15.096630 1505 raft_consensus.cc:1261] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Refusing update from remote peer 7e95b719833149158e6b28ef0ba5955a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:15.096627 1652 raft_consensus.cc:1261] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Refusing update from remote peer 7e95b719833149158e6b28ef0ba5955a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:15.097007 1731 consensus_queue.cc:1035] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:15.097151 1725 consensus_queue.cc:1035] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:15.099463 1735 mvcc.cc:205] Tried to move back new op lower bound from 7317552722312957952 to 7317552722223542272. Current Snapshot: MvccSnapshot[applied={T|T < 7317552722312957952}]
I20260812 05:18:15.099645 1737 mvcc.cc:205] Tried to move back new op lower bound from 7317552722312957952 to 7317552722223542272. Current Snapshot: MvccSnapshot[applied={T|T < 7317552722312957952}]
I20260812 05:18:15.099928 1738 mvcc.cc:205] Tried to move back new op lower bound from 7317552722312957952 to 7317552722223542272. Current Snapshot: MvccSnapshot[applied={T|T < 7317552722312957952}]
I20260812 05:18:15.101693 1208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:15.101799 1208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:15.102604 1208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Sending DeleteTablet for 3 replicas of tablet 814d353ecda94d61a5cec951b7415e7e
I20260812 05:18:15.102945 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1274
I20260812 05:18:15.102989 1630 tablet_service.cc:1514] Processing DeleteTablet for tablet 814d353ecda94d61a5cec951b7415e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:15 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59010
I20260812 05:18:15.103021 1342 tablet_service.cc:1514] Processing DeleteTablet for tablet 814d353ecda94d61a5cec951b7415e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:15 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:36882
I20260812 05:18:15.103564 1483 tablet_service.cc:1514] Processing DeleteTablet for tablet 814d353ecda94d61a5cec951b7415e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:15 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:60464
I20260812 05:18:15.103840 1750 tablet_replica.cc:330] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: stopping tablet replica
I20260812 05:18:15.103921 1750 raft_consensus.cc:2229] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:15.103988 1750 raft_consensus.cc:2258] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:15.104537 1750 ts_tablet_manager.cc:1905] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:15.105357 1749 tablet_replica.cc:330] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: stopping tablet replica
I20260812 05:18:15.105489 1749 raft_consensus.cc:2229] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:15.105623 1749 raft_consensus.cc:2258] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:15.106047 1749 ts_tablet_manager.cc:1905] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:15.106484 1750 ts_tablet_manager.cc:1918] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:15.106616 1750 log.cc:1196] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/814d353ecda94d61a5cec951b7415e7e
I20260812 05:18:15.106980 1750 ts_tablet_manager.cc:1939] T 814d353ecda94d61a5cec951b7415e7e P 02e07d591a8542f49b95ff3aef77eca7: Deleting consensus metadata
I20260812 05:18:15.107462 1191 catalog_manager.cc:4841] TS 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797): tablet 814d353ecda94d61a5cec951b7415e7e (table test-table [id=1331e72f910c4a43b22af8067fa9f214]) successfully deleted
I20260812 05:18:15.107882 1749 ts_tablet_manager.cc:1918] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:15.107947 1749 log.cc:1196] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/814d353ecda94d61a5cec951b7415e7e
I20260812 05:18:15.108148 1749 ts_tablet_manager.cc:1939] T 814d353ecda94d61a5cec951b7415e7e P 8729388ea95b4467a1881dd8d119cb1a: Deleting consensus metadata
I20260812 05:18:15.108572 1192 catalog_manager.cc:4841] TS 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969): tablet 814d353ecda94d61a5cec951b7415e7e (table test-table [id=1331e72f910c4a43b22af8067fa9f214]) successfully deleted
W20260812 05:18:15.110761 1192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:44397 (error 108)
I20260812 05:18:15.111033 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1416
I20260812 05:18:15.111316 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.111190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44397 (local address 127.0.0.1:36894)
0812 05:18:15.111234 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.111276 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:44397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 05:18:15.112171 1192 catalog_manager.cc:4584] TS 7e95b719833149158e6b28ef0ba5955a (127.18.166.193:44397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 814d353ecda94d61a5cec951b7415e7e: Network error: Client connection negotiation failed: client connection to 127.18.166.193:44397: connect: Connection refused (error 111)
I20260812 05:18:15.118188 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1558
I20260812 05:18:15.124469 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1175
I20260812 05:18:15.131299 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34219
--webserver_interface=127.18.166.254
--webserver_port=37109
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:34219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:15.215680 1752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:15.215883 1752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:15.215905 1752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:15.216970 1752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:15.217003 1752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:15.217017 1752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:15.217031 1752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:15.217042 1752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:15.218456 1752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:34219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=37109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:15.218652 1752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:15.218828 1752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:15.229002 1759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:15.229140 1752 server_base.cc:1034] running on GCE node
W20260812 05:18:15.229022 1762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:15.229022 1760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:15.229591 1752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:15.229941 1752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:15.231081 1752 hybrid_clock.cc:648] HybridClock initialized: now 1786511895231063 us; error 40 us; skew 500 ppm
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:15.233717 1752 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:15.234272 1752 webserver.cc:460] Webserver started at http://127.18.166.254:37109/ using document root <none> and password file <none>
I20260812 05:18:15.234529 1752 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:15.234614 1752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:15.236001 1752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:18:15.236652 1770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.236810 1752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:18:15.236907 1752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "873f064675b74400a3d30e8354cb6fe3"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c9b5576fb2d6eab66f0c803ca5eb96d0"
server_key_iv: "c53adf06664a4cd7c5fb01817f895696"
server_key_version: "encryptionkey@0"
I20260812 05:18:15.237288 1752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:15.255283 1752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:15.257505 1752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:15.257645 1752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:15.261659 1752 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:34219
I20260812 05:18:15.261718 1825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:34219 every 8 connection(s)
I20260812 05:18:15.262115 1752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:15.263835 1826 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:18:15.264555 1826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap starting.
I20260812 05:18:15.267298 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1752
I20260812 05:18:15.267575 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:44397
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:34219
--webserver_port=35681
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:15.274194 1826 log.cc:824] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:15.276160 1826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:15.276477 1826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap complete.
I20260812 05:18:15.278870 1826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:15.279125 1826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873f064675b74400a3d30e8354cb6fe3, State: Initialized, Role: FOLLOWER
I20260812 05:18:15.279331 1826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:15.279428 1826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:15.279486 1826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:15.279534 1826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:18:15.280539 1826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:15.280668 1826 leader_election.cc:302] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873f064675b74400a3d30e8354cb6fe3; no voters:
I20260812 05:18:15.280879 1826 leader_election.cc:288] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:18:15.280923 1830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:18:15.281092 1830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 LEADER]: Becoming Leader. State: Replica: 873f064675b74400a3d30e8354cb6fe3, State: Running, Role: LEADER
I20260812 05:18:15.281169 1826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:15.281241 1830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:15.281652 1831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "873f064675b74400a3d30e8354cb6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } } }
I20260812 05:18:15.281738 1831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:15.281965 1832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873f064675b74400a3d30e8354cb6fe3. Latest consensus state: current_term: 2 leader_uuid: "873f064675b74400a3d30e8354cb6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } } }
I20260812 05:18:15.282029 1832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:15.282121 1837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:15.282866 1837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:15.283162 1837 catalog_manager.cc:1259] Loaded cluster ID: d67e26e8b28541059aa50ff0ef81e4b4
I20260812 05:18:15.283226 1837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:15.283474 1837 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 05:18:15.288640 1837 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 05:18:15.291136 1837 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:15.291359 1837 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Loaded TSK: 0
I20260812 05:18:15.291572 1837 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:18:15.355602 1828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:15.355819 1828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:15.355845 1828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:15.357048 1828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:15.357093 1828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:15.357121 1828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:15.358681 1828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:44397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=35681
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:15.358944 1828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:15.359208 1828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:15.365166 1854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:15.365139 1855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:15.365147 1857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:15.365603 1828 server_base.cc:1034] running on GCE node
I20260812 05:18:15.365836 1828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:15.366079 1828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:15.367246 1828 hybrid_clock.cc:648] HybridClock initialized: now 1786511895367209 us; error 56 us; skew 500 ppm
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:15.369863 1828 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:15.370375 1828 webserver.cc:460] Webserver started at http://127.18.166.193:35681/ using document root <none> and password file <none>
I20260812 05:18:15.370631 1828 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:15.370689 1828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:15.372052 1828 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:15.372751 1865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.372910 1828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:18:15.373040 1828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "7e95b719833149158e6b28ef0ba5955a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "ee788ac64504f87845bed5c31d74c3cd"
server_key_iv: "f4f1368e024412b239122728b68a1238"
server_key_version: "encryptionkey@0"
I20260812 05:18:15.373385 1828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:15.393297 1828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:15.395540 1828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:15.395725 1828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:15.396005 1828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:15.396492 1872 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:18:15.398042 1828 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:18:15.398101 1828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 05:18:15.398131 1828 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:18:15.398816 1828 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:18:15.398864 1828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:18:15.398908 1872 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Bootstrap starting.
I20260812 05:18:15.406992 1828 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:44397
I20260812 05:18:15.407521 1828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:18:15.408156 1984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:44397 every 8 connection(s)
I20260812 05:18:15.411197 1872 log.cc:824] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:15.413681 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1828
I20260812 05:18:15.413949 1872 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:15.414003 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:38797
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:34219
--webserver_port=36653
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:15.414288 1872 tablet_bootstrap.cc:492] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Bootstrap complete.
I20260812 05:18:15.415514 1872 ts_tablet_manager.cc:1397] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:15.416932 1872 raft_consensus.cc:348] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.417373 1872 raft_consensus.cc:729] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e95b719833149158e6b28ef0ba5955a, State: Initialized, Role: FOLLOWER
I20260812 05:18:15.417606 1872 consensus_queue.cc:260] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.417907 1872 ts_tablet_manager.cc:1428] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260812 05:18:15.421476 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.408338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:59437)
0812 05:18:15.409660 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.409704 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.410302 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.410879 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.410883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.411253 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:15.411509 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.411515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.412684 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.412685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.413929 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.413932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.414795 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.414796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.414949 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.415601 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:15.415612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:15.417226 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:15.419748 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.419755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.419757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.419986 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.420321 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.420322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.420322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.420433 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.420937 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:15.420943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:15.421054 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.421149 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.421276 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"thread_start_us":111,"threads_started":1}
I20260812 05:18:15.421663 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.408383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:59437 (local address 127.18.166.254:34219)
0812 05:18:15.409560 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.409569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.409848 (+ 279us) server_negotiation.cc:407] Connection header received
0812 05:18:15.410570 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.410580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.410662 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.410834 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:15.411660 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.412546 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.414026 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.414663 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.414776 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.417340 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:15.417344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:15.417344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:15.417358 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:15.419640 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.420083 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.420086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.420087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.420222 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.420523 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.420524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.420524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.420848 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:15.420950 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.421235 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.421393 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":85,"threads_started":1}
I20260812 05:18:15.422493 1985 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34219
I20260812 05:18:15.422575 1985 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:15.422761 1985 heartbeater.cc:511] Master 127.18.166.254:34219 requested a full tablet report, sending...
I20260812 05:18:15.423521 1788 ts_manager.cc:194] Registered new tserver with Master: 7e95b719833149158e6b28ef0ba5955a (127.18.166.193:44397)
I20260812 05:18:15.424863 1788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:59437
I20260812 05:18:15.425982 1985 heartbeater.cc:503] Master 127.18.166.254:34219 was elected leader, sending a full tablet report...
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:18:15.433106 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.424268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44397 (local address 127.0.0.1:36910)
0812 05:18:15.424550 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.424582 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.424698 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.424893 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.424899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.425215 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:15.425276 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.425279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.426508 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.426509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.427528 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.427532 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.427983 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.427984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.428066 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.428529 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:15.428534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:15.430069 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:15.432025 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.432031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.432033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.432241 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.432463 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.432464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.432464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.432526 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.432829 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:15.432835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:15.432878 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.432942 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.433033 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
I20260812 05:18:15.433202 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.424323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.18.166.193:44397)
0812 05:18:15.424482 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.424487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.424692 (+ 205us) server_negotiation.cc:407] Connection header received
0812 05:18:15.424789 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.424792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.424813 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.424889 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:15.425396 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.426425 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.427634 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.427908 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.427960 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.430154 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:15.430163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:15.430163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:15.430171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:15.431925 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.432324 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.432327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.432327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.432392 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.432589 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.432590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.432591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.432763 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:15.432844 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.432993 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.433092 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260812 05:18:15.433600 1916 tablet_service.cc:1514] Processing DeleteTablet for tablet 814d353ecda94d61a5cec951b7415e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:15 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:36910
I20260812 05:18:15.434230 1995 tablet_replica.cc:330] stopping tablet replica
I20260812 05:18:15.434365 1995 raft_consensus.cc:2229] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:15.434449 1995 raft_consensus.cc:2258] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:15.434800 1995 ts_tablet_manager.cc:1905] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:15.436901 1995 ts_tablet_manager.cc:1918] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:15.437011 1995 log.cc:1196] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/814d353ecda94d61a5cec951b7415e7e
I20260812 05:18:15.437280 1995 ts_tablet_manager.cc:1939] T 814d353ecda94d61a5cec951b7415e7e P 7e95b719833149158e6b28ef0ba5955a: Deleting consensus metadata
I20260812 05:18:15.437884 1772 catalog_manager.cc:4841] TS 7e95b719833149158e6b28ef0ba5955a (127.18.166.193:44397): tablet 814d353ecda94d61a5cec951b7415e7e (table test-table [id=1331e72f910c4a43b22af8067fa9f214]) successfully deleted
W20260812 05:18:15.503935 1991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:15.504160 1991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:15.504184 1991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:15.505249 1991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:15.505285 1991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:15.505311 1991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:15.506793 1991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:38797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=36653
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:15.506971 1991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:15.507145 1991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:15.512050 2001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:15.512079 2003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:15.512050 2000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:15.512334 1991 server_base.cc:1034] running on GCE node
I20260812 05:18:15.512553 1991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:15.512827 1991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:15.513984 1991 hybrid_clock.cc:648] HybridClock initialized: now 1786511895513972 us; error 30 us; skew 500 ppm
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:15.516531 1991 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:15.516961 1991 webserver.cc:460] Webserver started at http://127.18.166.194:36653/ using document root <none> and password file <none>
I20260812 05:18:15.517194 1991 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:15.517249 1991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:15.518766 1991 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:15.519377 2010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.519553 1991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:18:15.519655 1991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "02e07d591a8542f49b95ff3aef77eca7"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "05b653673d59b0ab8f46a37ad332978c"
server_key_iv: "d9a461655738ae1ba1f02086ef682cdd"
server_key_version: "encryptionkey@0"
I20260812 05:18:15.520001 1991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:15.551993 1991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:15.554016 1991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:15.554199 1991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:15.554456 1991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:15.554777 1991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:15.554832 1991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.554874 1991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:15.554904 1991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.560611 1991 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38797
I20260812 05:18:15.560683 2134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38797 every 8 connection(s)
I20260812 05:18:15.561017 1991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:18:15.562095 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 1991
I20260812 05:18:15.562381 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:33969
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:34219
--webserver_port=42415
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:15.571074 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.561766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:50737 (local address 127.18.166.254:34219)
0812 05:18:15.561841 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.561843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.562075 (+ 232us) server_negotiation.cc:407] Connection header received
0812 05:18:15.562664 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.562665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.562682 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.562759 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:15.563675 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.564266 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.565243 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.565629 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.565691 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.568110 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:15.568112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:15.568112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:15.568121 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:15.569610 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.570036 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.570038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.570038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.570095 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.570353 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.570354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.570354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.570467 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:15.570544 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.570880 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.570981 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:18:15.571238 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.561765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:50737)
0812 05:18:15.561936 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.561978 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.562554 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.562811 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.562814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.563379 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:15.563570 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.563574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.564360 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.564360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.565149 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.565151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.565718 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.565720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.565867 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.566392 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:15.566403 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:15.568005 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:15.569697 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.569704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.569706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.569937 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.570163 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.570163 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.570164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.570277 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.570533 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:15.570541 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:15.570731 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.570846 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.571003 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":108,"threads_started":1}
I20260812 05:18:15.571758 2135 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34219
I20260812 05:18:15.571875 2135 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:15.572055 2135 heartbeater.cc:511] Master 127.18.166.254:34219 requested a full tablet report, sending...
I20260812 05:18:15.572479 1788 ts_manager.cc:194] Registered new tserver with Master: 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797)
I20260812 05:18:15.572866 1788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:50737
W20260812 05:18:15.646859 2138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:15.647075 2138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:15.647107 2138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:15.648312 2138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:15.648373 2138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:15.648420 2138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:15.649996 2138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:33969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=42415
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:15.650252 2138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:15.650490 2138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:15.655556 2145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:15.655625 2144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:15.655858 2138 server_base.cc:1034] running on GCE node
W20260812 05:18:15.655575 2147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:15.656106 2138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:15.656351 2138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:15.657473 2138 hybrid_clock.cc:648] HybridClock initialized: now 1786511895657471 us; error 42 us; skew 500 ppm
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:15.660147 2138 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:15.660598 2138 webserver.cc:460] Webserver started at http://127.18.166.195:42415/ using document root <none> and password file <none>
I20260812 05:18:15.660835 2138 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:15.660921 2138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:15.662358 2138 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:15.662945 2154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.663160 2138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:15.663260 2138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "8729388ea95b4467a1881dd8d119cb1a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c640919f257d135a709700ea0c854899"
server_key_iv: "8559d488a5174211c65bfe3fce177590"
server_key_version: "encryptionkey@0"
I20260812 05:18:15.663606 2138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:15.678807 2138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:15.681255 2138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:15.681480 2138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:15.681769 2138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:15.682113 2138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:15.682158 2138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.682201 2138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:15.682229 2138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:15.688279 2138 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:33969
I20260812 05:18:15.688318 2275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:33969 every 8 connection(s)
I20260812 05:18:15.688686 2138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:18:15.689009 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 2138
Aug 12 05:18:15 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511895, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:15.696854 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.689480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52208 (local address 127.18.166.254:34219)
0812 05:18:15.689536 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.689538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.689733 (+ 195us) server_negotiation.cc:407] Connection header received
0812 05:18:15.689790 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.689791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.689803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.689871 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:15.690468 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.691283 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.692262 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.692508 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.692584 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.694077 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:15.694079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:15.694079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:15.694087 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:15.695652 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.696039 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.696040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.696040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.696090 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.696334 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.696335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.696335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.696439 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:15.696569 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.696662 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.696743 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:18:15.699649 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.689500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:47065 (local address 127.18.166.254:34219)
0812 05:18:15.689593 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.689595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.689891 (+ 296us) server_negotiation.cc:407] Connection header received
0812 05:18:15.690398 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.690399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.690411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.690481 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:15.691625 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.692343 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.693623 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.693953 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.694062 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.697083 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:15.697084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:15.697084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:15.697092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:15.698532 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.698902 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.698903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.698903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.698955 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:15.699128 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:15.699129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:15.699129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:15.699224 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:15.699271 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.699516 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.699584 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260812 05:18:15.699802 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.689552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:47065)
0812 05:18:15.689733 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.689788 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.690295 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.690553 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.690557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.691290 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:15.691529 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.691535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.692422 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.692423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.693530 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.693533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.694071 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.694072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.694217 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.694739 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:15.694749 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:15.696970 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:15.698605 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.698612 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.698614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.698835 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.698994 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:15.698995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:15.698995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:15.699078 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:15.699289 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:15.699295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:15.699405 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.699488 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.699605 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260812 05:18:15.700249 2276 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34219
I20260812 05:18:15.700337 2276 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:15.700546 2276 heartbeater.cc:511] Master 127.18.166.254:34219 requested a full tablet report, sending...
I20260812 05:18:15.700896 1788 ts_manager.cc:194] Registered new tserver with Master: 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
I20260812 05:18:15.701273 1788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:47065
I20260812 05:18:15.708871 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:18:15.712106 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.709291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52212 (local address 127.18.166.254:34219)
0812 05:18:15.709376 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.709378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.709537 (+ 159us) server_negotiation.cc:407] Connection header received
0812 05:18:15.709641 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.709642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.709692 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.709782 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:15.709933 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.710462 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.711247 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.711468 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.711568 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.711756 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:15.711864 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:18:15.711962 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.711987 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.712013 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 05:18:15.712419 1788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:18:15.714185 1788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:18:15.716698 1916 tablet_service.cc:1467] Processing CreateTablet for tablet 4343316660604ebfa9656b8f060af7ba (DEFAULT_TABLE table=test-table [id=4f20c9cd781149a2bb9bb2a8ac9eb0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:15.716914 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4343316660604ebfa9656b8f060af7ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:15.718438 1872 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Bootstrap starting.
I20260812 05:18:15.718788 1872 tablet_bootstrap.cc:654] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:15.719272 1872 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: No bootstrap required, opened a new log
I20260812 05:18:15.719327 1872 ts_tablet_manager.cc:1397] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:15.719676 1872 raft_consensus.cc:348] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.719758 1872 raft_consensus.cc:374] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:15.719816 1872 raft_consensus.cc:729] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e95b719833149158e6b28ef0ba5955a, State: Initialized, Role: FOLLOWER
I20260812 05:18:15.719864 1872 consensus_queue.cc:260] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.719962 1872 ts_tablet_manager.cc:1428] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:15.720247 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.716005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33969 (local address 127.0.0.1:59012)
0812 05:18:15.716250 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.716261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.716304 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.716550 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.716552 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.716554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:15.716612 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.716614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.717983 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.717984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.718961 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.718964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.719991 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.719993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.720060 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.720085 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.720119 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.720158 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":26,"thread_start_us":92,"threads_started":1}
I20260812 05:18:15.720350 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.716005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38797 (local address 127.0.0.1:60480)
0812 05:18:15.716074 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.716086 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.716138 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.716640 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.716641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.716642 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:15.716685 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.716687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.718148 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.718149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.719180 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.719184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.720099 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.720100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.720164 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.720167 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.720215 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.720251 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:18:15.720372 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.716258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.18.166.194:38797)
0812 05:18:15.716472 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.716476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.716485 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:18:15.716542 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.716544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.716559 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.716643 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:15.716768 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.718051 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.719295 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.720023 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.720104 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.720163 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.720244 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.720291 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":114,"threads_started":1}
I20260812 05:18:15.720671 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.716146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.18.166.195:33969)
0812 05:18:15.716360 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.716366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.716377 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:18:15.716419 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.716421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.716436 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.716528 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:15.716701 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.717867 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.719293 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.719887 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.720424 (+ 537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.720505 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.720534 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.720593 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260812 05:18:15.721117 2061 tablet_service.cc:1467] Processing CreateTablet for tablet 4343316660604ebfa9656b8f060af7ba (DEFAULT_TABLE table=test-table [id=4f20c9cd781149a2bb9bb2a8ac9eb0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:15.721494 2061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4343316660604ebfa9656b8f060af7ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:15.721386 2203 tablet_service.cc:1467] Processing CreateTablet for tablet 4343316660604ebfa9656b8f060af7ba (DEFAULT_TABLE table=test-table [id=4f20c9cd781149a2bb9bb2a8ac9eb0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:15.721745 2203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4343316660604ebfa9656b8f060af7ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:15.724380 2287 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Bootstrap starting.
I20260812 05:18:15.724767 2288 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Bootstrap starting.
I20260812 05:18:15.725030 2287 tablet_bootstrap.cc:654] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:15.725348 2288 tablet_bootstrap.cc:654] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:15.725613 2287 log.cc:824] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:15.725898 2288 log.cc:824] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:15.726387 2287 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: No bootstrap required, opened a new log
I20260812 05:18:15.726454 2287 ts_tablet_manager.cc:1397] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:18:15.726660 2288 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: No bootstrap required, opened a new log
I20260812 05:18:15.726737 2288 ts_tablet_manager.cc:1397] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:18:15.727910 2287 raft_consensus.cc:348] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.728101 2287 raft_consensus.cc:374] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:15.728156 2287 raft_consensus.cc:729] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e07d591a8542f49b95ff3aef77eca7, State: Initialized, Role: FOLLOWER
I20260812 05:18:15.728214 2288 raft_consensus.cc:348] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.728328 2287 consensus_queue.cc:260] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.728389 2288 raft_consensus.cc:374] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:15.728435 2288 raft_consensus.cc:729] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8729388ea95b4467a1881dd8d119cb1a, State: Initialized, Role: FOLLOWER
I20260812 05:18:15.728540 2288 consensus_queue.cc:260] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.728606 2135 heartbeater.cc:503] Master 127.18.166.254:34219 was elected leader, sending a full tablet report...
I20260812 05:18:15.728605 2287 ts_tablet_manager.cc:1428] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:18:15.728809 2288 ts_tablet_manager.cc:1428] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:15.728937 2276 heartbeater.cc:503] Master 127.18.166.254:34219 was elected leader, sending a full tablet report...
W20260812 05:18:15.811594 2136 tablet.cc:2369] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:15.853046 1992 raft_consensus.cc:482] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:15.853250 1992 raft_consensus.cc:504] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.853667 1992 leader_election.cc:288] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
I20260812 05:18:15.856534 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.853846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38797 (local address 127.18.166.193:48341)
0812 05:18:15.853911 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.853922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.853978 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.854127 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.854128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.854130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:15.854205 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.854207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.854866 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.854867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.855646 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.855648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.856228 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.856229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.856321 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.856340 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.856410 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.856461 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":9}
I20260812 05:18:15.856530 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.853934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:48341 (local address 127.18.166.194:38797)
0812 05:18:15.853993 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.853995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.854029 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:18:15.854061 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.854062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.854075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.854122 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:15.854291 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.854775 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.855748 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.856139 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.856229 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.856239 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.856417 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.856459 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:18:15.856650 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.853896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33969 (local address 127.18.166.193:51345)
0812 05:18:15.854075 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.854090 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:15.854167 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:15.854324 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:15.854324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:15.854326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:15.854412 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.854413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.855034 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.855035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.855784 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:15.855786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.856352 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.856354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:15.856447 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.856450 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:18:15.856517 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:18:15.856555 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":96,"threads_started":1}
I20260812 05:18:15.856899 2084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343316660604ebfa9656b8f060af7ba" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02e07d591a8542f49b95ff3aef77eca7" is_pre_election: true
I20260812 05:18:15.857030 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.854025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:51345 (local address 127.18.166.195:33969)
0812 05:18:15.854081 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.854083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.854160 (+ 77us) server_negotiation.cc:407] Connection header received
0812 05:18:15.854227 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.854228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.854237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.854320 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:15.854483 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.854947 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.855877 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.856252 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.856339 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.856350 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.856525 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.856565 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:18:15.857072 2084 raft_consensus.cc:2454] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 0.
I20260812 05:18:15.857322 1866 leader_election.cc:302] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02e07d591a8542f49b95ff3aef77eca7, 7e95b719833149158e6b28ef0ba5955a; no voters:
I20260812 05:18:15.857303 2225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343316660604ebfa9656b8f060af7ba" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8729388ea95b4467a1881dd8d119cb1a" is_pre_election: true
I20260812 05:18:15.857477 2225 raft_consensus.cc:2454] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 0.
I20260812 05:18:15.857473 1992 raft_consensus.cc:2781] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:18:15.857546 1992 raft_consensus.cc:482] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:18:15.857576 1992 raft_consensus.cc:3037] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:15.858287 1992 raft_consensus.cc:504] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.858472 1992 leader_election.cc:288] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 election: Requested vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
I20260812 05:18:15.858665 2084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343316660604ebfa9656b8f060af7ba" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02e07d591a8542f49b95ff3aef77eca7"
I20260812 05:18:15.858666 2225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343316660604ebfa9656b8f060af7ba" candidate_uuid: "7e95b719833149158e6b28ef0ba5955a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8729388ea95b4467a1881dd8d119cb1a"
I20260812 05:18:15.858754 2084 raft_consensus.cc:3037] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:15.858753 2225 raft_consensus.cc:3037] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:15.859715 2225 raft_consensus.cc:2454] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 1.
I20260812 05:18:15.859715 2084 raft_consensus.cc:2454] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e95b719833149158e6b28ef0ba5955a in term 1.
I20260812 05:18:15.859948 1867 leader_election.cc:302] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e95b719833149158e6b28ef0ba5955a, 8729388ea95b4467a1881dd8d119cb1a; no voters:
I20260812 05:18:15.860069 1992 raft_consensus.cc:2781] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:15.860225 1992 raft_consensus.cc:686] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 LEADER]: Becoming Leader. State: Replica: 7e95b719833149158e6b28ef0ba5955a, State: Running, Role: LEADER
I20260812 05:18:15.860317 1992 consensus_queue.cc:237] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:15.861171 1788 catalog_manager.cc:5495] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e95b719833149158e6b28ef0ba5955a (127.18.166.193). New cstate: current_term: 1 leader_uuid: "7e95b719833149158e6b28ef0ba5955a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } health_report { overall_health: HEALTHY } } }
I20260812 05:18:15.901556 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.899105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36912 (local address 127.18.166.193:44397)
0812 05:18:15.899170 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:15.899172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:15.899180 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:18:15.899223 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:15.899224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:15.899240 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:15.899319 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:15.899451 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.899959 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.900691 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:15.900928 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:15.901008 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:15.901174 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:15.901286 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:18:15.901372 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:15.901434 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 05:18:15.901460 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:18:15.903244 2225 raft_consensus.cc:1261] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Refusing update from remote peer 7e95b719833149158e6b28ef0ba5955a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:15.903249 2084 raft_consensus.cc:1261] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Refusing update from remote peer 7e95b719833149158e6b28ef0ba5955a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:15.903604 1992 consensus_queue.cc:1035] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:15.903730 2294 consensus_queue.cc:1035] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [LEADER]: Connected to new peer: Peer: permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:15.906824 2298 mvcc.cc:205] Tried to move back new op lower bound from 7317552725616996352 to 7317552725444521984. Current Snapshot: MvccSnapshot[applied={T|T < 7317552725616996352}]
I20260812 05:18:15.907475 2297 mvcc.cc:205] Tried to move back new op lower bound from 7317552725616996352 to 7317552725444521984. Current Snapshot: MvccSnapshot[applied={T|T < 7317552725616996352 or (T in {7317552725616996352})}]
I20260812 05:18:15.908475 1787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:15.908574 1787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:15.909349 1787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Sending DeleteTablet for 3 replicas of tablet 4343316660604ebfa9656b8f060af7ba
I20260812 05:18:15.909695 2061 tablet_service.cc:1514] Processing DeleteTablet for tablet 4343316660604ebfa9656b8f060af7ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:15 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:60480
I20260812 05:18:15.909746 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1828
I20260812 05:18:15.909749 2203 tablet_service.cc:1514] Processing DeleteTablet for tablet 4343316660604ebfa9656b8f060af7ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:15 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:59012
I20260812 05:18:15.911748 2306 tablet_replica.cc:330] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: stopping tablet replica
I20260812 05:18:15.911906 2306 raft_consensus.cc:2229] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:15.912067 2306 raft_consensus.cc:2258] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:15.912595 2306 ts_tablet_manager.cc:1905] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:15.914391 2306 ts_tablet_manager.cc:1918] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:15.914492 2306 log.cc:1196] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/4343316660604ebfa9656b8f060af7ba
I20260812 05:18:15.914762 2306 ts_tablet_manager.cc:1939] T 4343316660604ebfa9656b8f060af7ba P 02e07d591a8542f49b95ff3aef77eca7: Deleting consensus metadata
I20260812 05:18:15.915243 1771 catalog_manager.cc:4841] TS 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797): tablet 4343316660604ebfa9656b8f060af7ba (table test-table [id=4f20c9cd781149a2bb9bb2a8ac9eb0bb]) successfully deleted
I20260812 05:18:15.915370 2307 tablet_replica.cc:330] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: stopping tablet replica
I20260812 05:18:15.915505 2307 raft_consensus.cc:2229] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:15.915635 2307 raft_consensus.cc:2258] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:15.916098 2307 ts_tablet_manager.cc:1905] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:15.918115 2307 ts_tablet_manager.cc:1918] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:15.918201 2307 log.cc:1196] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/4343316660604ebfa9656b8f060af7ba
I20260812 05:18:15.918493 2307 ts_tablet_manager.cc:1939] T 4343316660604ebfa9656b8f060af7ba P 8729388ea95b4467a1881dd8d119cb1a: Deleting consensus metadata
I20260812 05:18:15.918957 1772 catalog_manager.cc:4841] TS 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969): tablet 4343316660604ebfa9656b8f060af7ba (table test-table [id=4f20c9cd781149a2bb9bb2a8ac9eb0bb]) successfully deleted
W20260812 05:18:15.920125 1772 connection.cc:535] client connection to 127.18.166.193:44397 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 05:18:15.920243 1772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:18:15.920627 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1991
I20260812 05:18:15.920753 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:15.920593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:44397 (local address 127.0.0.1:36928)
0812 05:18:15.920655 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:15.920698 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:44397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 05:18:15.921690 1772 catalog_manager.cc:4584] TS 7e95b719833149158e6b28ef0ba5955a (127.18.166.193:44397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4343316660604ebfa9656b8f060af7ba: Network error: Client connection negotiation failed: client connection to 127.18.166.193:44397: connect: Connection refused (error 111)
I20260812 05:18:15.929265 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 2138
I20260812 05:18:15.934337 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 1752
I20260812 05:18:15.941032 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34219
--webserver_interface=127.18.166.254
--webserver_port=37109
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:34219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:16.026736 2309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:16.026974 2309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:16.026998 2309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:16.028183 2309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:16.028234 2309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:16.028249 2309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:16.028262 2309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:16.028317 2309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:16.029834 2309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:34219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=37109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:16.030093 2309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:16.030387 2309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:16.035375 2315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.035370 2317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.035416 2314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:16.035724 2309 server_base.cc:1034] running on GCE node
I20260812 05:18:16.035992 2309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:16.036299 2309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:16.037480 2309 hybrid_clock.cc:648] HybridClock initialized: now 1786511896037471 us; error 37 us; skew 500 ppm
Aug 12 05:18:16 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511896, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:16.040045 2309 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:16.040501 2309 webserver.cc:460] Webserver started at http://127.18.166.254:37109/ using document root <none> and password file <none>
I20260812 05:18:16.040743 2309 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:16.040815 2309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:16.042285 2309 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:18:16.042968 2325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.043138 2309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:18:16.043237 2309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "873f064675b74400a3d30e8354cb6fe3"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c9b5576fb2d6eab66f0c803ca5eb96d0"
server_key_iv: "c53adf06664a4cd7c5fb01817f895696"
server_key_version: "encryptionkey@0"
I20260812 05:18:16.043601 2309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:16.070928 2309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:16.072940 2309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:16.073120 2309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:16.077584 2309 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:34219
I20260812 05:18:16.077641 2380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:34219 every 8 connection(s)
I20260812 05:18:16.078081 2309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:16.079670 2381 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:18:16.080262 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap starting.
I20260812 05:18:16.087507 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 2309
I20260812 05:18:16.087836 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:44397
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:34219
--webserver_port=35681
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:16.089713 2381 log.cc:824] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:16.092031 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:16.092375 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3: Bootstrap complete.
I20260812 05:18:16.094869 2381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:16.095098 2381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873f064675b74400a3d30e8354cb6fe3, State: Initialized, Role: FOLLOWER
I20260812 05:18:16.095281 2381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:16.095345 2381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:16.095408 2381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:16.095458 2381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 2 FOLLOWER]: Advancing to term 3
I20260812 05:18:16.096490 2381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:16.096577 2381 leader_election.cc:302] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873f064675b74400a3d30e8354cb6fe3; no voters:
I20260812 05:18:16.096801 2381 leader_election.cc:288] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 05:18:16.096854 2385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 05:18:16.097025 2385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [term 3 LEADER]: Becoming Leader. State: Replica: 873f064675b74400a3d30e8354cb6fe3, State: Running, Role: LEADER
I20260812 05:18:16.097091 2381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:16.097163 2385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } }
I20260812 05:18:16.097476 2385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873f064675b74400a3d30e8354cb6fe3. Latest consensus state: current_term: 3 leader_uuid: "873f064675b74400a3d30e8354cb6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } } }
I20260812 05:18:16.097611 2385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:16.097482 2388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "873f064675b74400a3d30e8354cb6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873f064675b74400a3d30e8354cb6fe3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34219 } } }
I20260812 05:18:16.097749 2388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873f064675b74400a3d30e8354cb6fe3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:16.097903 2397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:16.098971 2397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:16.099092 2397 catalog_manager.cc:1259] Loaded cluster ID: d67e26e8b28541059aa50ff0ef81e4b4
I20260812 05:18:16.099195 2397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 05:18:16.099671 2397 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f38426719ad google::LogMessage::Fail() at ??:0
@ 0x7f384267488a google::LogMessage::SendToLog() at ??:0
@ 0x7f38426714de google::LogMessage::Flush() at ??:0
@ 0x7f3842672a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f384a5920f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f384a59b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f384453f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3844534954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f38453986db start_thread at ??:0
@ 0x7f384116f61f clone at ??:0
W20260812 05:18:16.191061 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:16.191282 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:16.191309 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:16.192479 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:16.192523 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:16.192551 2383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:16.194067 2383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:44397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=35681
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:16.194320 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:16.194576 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:16.200843 2428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.200834 2426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.200830 2425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:16.201272 2383 server_base.cc:1034] running on GCE node
I20260812 05:18:16.201536 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:16.201844 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:16.203032 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786511896203021 us; error 29 us; skew 500 ppm
Aug 12 05:18:16 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511896, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:16.205750 2383 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:16.206198 2383 webserver.cc:460] Webserver started at http://127.18.166.193:35681/ using document root <none> and password file <none>
I20260812 05:18:16.206431 2383 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:16.206502 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:16.208038 2383 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:18:16.208799 2435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.209002 2383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:16.209102 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "7e95b719833149158e6b28ef0ba5955a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "ee788ac64504f87845bed5c31d74c3cd"
server_key_iv: "f4f1368e024412b239122728b68a1238"
server_key_version: "encryptionkey@0"
I20260812 05:18:16.209539 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:16.221136 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:16.223390 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:16.223568 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:16.223829 2383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:16.224318 2442 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:18:16.226303 2383 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:18:16.226372 2383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 05:18:16.226429 2383 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:18:16.227113 2383 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:18:16.227169 2383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:18:16.227248 2442 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Bootstrap starting.
I20260812 05:18:16.234850 2383 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:44397
I20260812 05:18:16.235013 2556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:44397 every 8 connection(s)
I20260812 05:18:16.235322 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:18:16.239039 2442 log.cc:824] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:16.241807 2442 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:16.242159 2442 tablet_bootstrap.cc:492] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Bootstrap complete.
I20260812 05:18:16.243508 2442 ts_tablet_manager.cc:1397] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260812 05:18:16.243777 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 2383
I20260812 05:18:16.244045 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:38797
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:34219
--webserver_port=36653
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:16.244966 2442 raft_consensus.cc:348] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:16.245328 2442 raft_consensus.cc:729] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e95b719833149158e6b28ef0ba5955a, State: Initialized, Role: FOLLOWER
I20260812 05:18:16.245532 2442 consensus_queue.cc:260] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:16.245834 2442 ts_tablet_manager.cc:1428] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260812 05:18:16.293584 2559 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:18:16.236153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:35199)
0812 05:18:16.236356 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.236404 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:16.236886 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:16.293325 (+ 56439us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":123,"threads_started":1}
W20260812 05:18:16.293859 2557 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:34219 (0 consecutive failures): Network error: Failed to ping master at 127.18.166.254:34219: Client connection negotiation failed: client connection to 127.18.166.254:34219: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:18:16.294226 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.294108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:56159)
0812 05:18:16.294166 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.294208 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:18:16.294574 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.294492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:40769)
0812 05:18:16.294535 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.294559 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 05:18:16.294724 2557 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:18:16.329655 2561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:16.329885 2561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:16.329922 2561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:16.331200 2561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:16.331262 2561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:16.331308 2561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:16.332794 2561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:38797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=36653
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:16.333065 2561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:16.333303 2561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:16.338449 2568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.338464 2567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:16.338488 2561 server_base.cc:1034] running on GCE node
W20260812 05:18:16.338470 2570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:16.338845 2561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:16.339089 2561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:16.340260 2561 hybrid_clock.cc:648] HybridClock initialized: now 1786511896340244 us; error 40 us; skew 500 ppm
Aug 12 05:18:16 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511896, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:16.342928 2561 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:16.343381 2561 webserver.cc:460] Webserver started at http://127.18.166.194:36653/ using document root <none> and password file <none>
I20260812 05:18:16.343643 2561 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:16.343714 2561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:16.345099 2561 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:16.345806 2577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.345995 2561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:18:16.346094 2561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "02e07d591a8542f49b95ff3aef77eca7"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "05b653673d59b0ab8f46a37ad332978c"
server_key_iv: "d9a461655738ae1ba1f02086ef682cdd"
server_key_version: "encryptionkey@0"
I20260812 05:18:16.346446 2561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:16.357685 2561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:16.359683 2561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:16.359864 2561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:16.360116 2561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:16.360436 2561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:16.360492 2561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.360530 2561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:16.360558 2561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.366660 2561 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38797
I20260812 05:18:16.366751 2699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38797 every 8 connection(s)
I20260812 05:18:16.367097 2561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:18:16.368197 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.367779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:53713)
0812 05:18:16.367938 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.368053 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
W20260812 05:18:16.368611 2700 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:34219 (0 consecutive failures): Network error: Failed to ping master at 127.18.166.254:34219: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
I20260812 05:18:16.368871 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.368783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:56987)
0812 05:18:16.368829 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.368854 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:18:16.369110 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.369055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:53267)
0812 05:18:16.369076 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.369099 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260812 05:18:16.369217 2700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:18:16.370549 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 2561
I20260812 05:18:16.370828 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:33969
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:34219
--webserver_port=42415
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:35395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:16.453465 2703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:16.453692 2703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:16.453718 2703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:16.454855 2703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:16.454891 2703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:16.454918 2703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:16.456328 2703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:35395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:33969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=42415
--tserver_master_addrs=127.18.166.254:34219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:16.456513 2703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:16.456704 2703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:16.461795 2709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.461843 2710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:16.461797 2712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:16.462023 2703 server_base.cc:1034] running on GCE node
I20260812 05:18:16.462299 2703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:16.462556 2703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:16.463725 2703 hybrid_clock.cc:648] HybridClock initialized: now 1786511896463715 us; error 40 us; skew 500 ppm
Aug 12 05:18:16 dist-test-slave-srgv krb5kdc[1160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511896, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:16.466295 2703 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:16.466727 2703 webserver.cc:460] Webserver started at http://127.18.166.195:42415/ using document root <none> and password file <none>
I20260812 05:18:16.466966 2703 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:16.467036 2703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:16.468410 2703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:16.469012 2719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.469190 2703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:18:16.469290 2703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "8729388ea95b4467a1881dd8d119cb1a"
format_stamp: "Formatted at 2026-08-12 05:18:14 on dist-test-slave-srgv"
server_key: "c640919f257d135a709700ea0c854899"
server_key_iv: "8559d488a5174211c65bfe3fce177590"
server_key_version: "encryptionkey@0"
I20260812 05:18:16.469703 2703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:16.479982 2703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:16.481905 2703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:16.482072 2703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:16.482329 2703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:16.482636 2703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:16.482692 2703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.482733 2703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:16.482762 2703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:16.488675 2703 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:33969
I20260812 05:18:16.488760 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:33969 every 8 connection(s)
I20260812 05:18:16.489125 2703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:18:16.490235 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.489851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:50853)
0812 05:18:16.490017 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.490093 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
W20260812 05:18:16.490653 2844 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:34219 (0 consecutive failures): Network error: Failed to ping master at 127.18.166.254:34219: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
I20260812 05:18:16.490962 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.490868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:52649)
0812 05:18:16.490918 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.490944 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:18:16.491215 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:16.491166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:35897)
0812 05:18:16.491186 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:16.491203 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260812 05:18:16.491345 2844 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:18:16.496037 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 2703
I20260812 05:18:17.295408 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.295153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:55837)
0812 05:18:17.295296 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:17.295341 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260812 05:18:17.369801 2849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.369607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:38327)
0812 05:18:17.369705 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:17.369736 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":58,"threads_started":1}
I20260812 05:18:17.491905 2850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.491692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:55993)
0812 05:18:17.491812 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:17.491854 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260812 05:18:17.721975 2851 raft_consensus.cc:482] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:17.722108 2851 raft_consensus.cc:504] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:17.722433 2851 leader_election.cc:288] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
Aug 12 05:18:17 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511896, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.195@KRBTEST.COM
Aug 12 05:18:17 dist-test-slave-srgv krb5kdc[1160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511896, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.194@KRBTEST.COM
I20260812 05:18:17.733273 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.722884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:58531 (local address 127.18.166.195:33969)
0812 05:18:17.723033 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:17.723041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:17.723066 (+ 25us) server_negotiation.cc:407] Connection header received
0812 05:18:17.723485 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:17.723487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:17.723523 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:17.723683 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:17.724503 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.725359 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.726542 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.726986 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.727118 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:17.729668 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:17.729672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:17.729673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:17.729687 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:17.731523 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:17.731958 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:17.731962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:17.731962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:17.732062 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:17.732340 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:17.732341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:17.732341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:17.732595 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:17.732689 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:17.732999 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 05:18:17.733131 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":54,"threads_started":1}
I20260812 05:18:17.733310 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.722663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:33969 (local address 127.18.166.193:58531)
0812 05:18:17.722734 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:17.722750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:17.722830 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:17.723746 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:17.723749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:17.724150 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:17.724340 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:17.724346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.725508 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.725509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:17.726463 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:17.726466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.727093 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.727093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:17.727213 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:17.727902 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:17.727906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:17.729542 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:17.731616 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:17.731624 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:17.731626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:17.731862 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:17.732140 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:17.732140 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:17.732141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:17.732262 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:17.732681 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:17.732688 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:17.732829 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 05:18:17.732932 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 05:18:17.733129 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:18:17.733366 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.722808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:49121 (local address 127.18.166.194:38797)
0812 05:18:17.722964 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:17.722982 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:17.723021 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:18:17.723477 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:17.723479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:17.723522 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:17.723700 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:17.724499 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.725360 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.726580 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.726986 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.727120 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:17.730004 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:17.730007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:17.730007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:17.730018 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:17.731885 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:17.732301 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:17.732304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:17.732304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:17.732398 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:17.732627 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:17.732628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:17.732629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:17.732854 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:17.732945 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:17.733114 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 05:18:17.733246 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260812 05:18:17.733774 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:17.722684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38797 (local address 127.18.166.193:49121)
0812 05:18:17.722798 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:17.722810 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:17.722869 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:17.723746 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:17.723749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:17.724150 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:17.724340 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:17.724346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.725532 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.725533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:17.726473 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:17.726476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:17.727100 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:17.727101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:17.727213 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:17.727838 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:17.727855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:17.729876 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:17.732008 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:17.732009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:17.732009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:17.732199 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:17.732475 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:17.732476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:17.732476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:17.732548 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:17.732928 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:17.732929 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:17.732977 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 05:18:17.733049 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:18:17.733199 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":78,"thread_start_us":63,"threads_started":1}
W20260812 05:18:17.734505 2436 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
W20260812 05:18:17.734611 2437 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
I20260812 05:18:17.734655 2437 leader_election.cc:302] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e95b719833149158e6b28ef0ba5955a; no voters: 02e07d591a8542f49b95ff3aef77eca7, 8729388ea95b4467a1881dd8d119cb1a
I20260812 05:18:17.734772 2851 raft_consensus.cc:2726] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:18:18.296272 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:18.296018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:39171)
0812 05:18:18.296174 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:18.296221 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260812 05:18:18.370604 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:18.370349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:57141)
0812 05:18:18.370510 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:18.370558 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260812 05:18:18.492624 2858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:18.492406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:36783)
0812 05:18:18.492544 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:18.492583 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260812 05:18:19.297097 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:19.296857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:56463)
0812 05:18:19.296995 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:19.297042 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260812 05:18:19.371461 2860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:19.371221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:54727)
0812 05:18:19.371373 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:19.371418 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260812 05:18:19.493432 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:19.493164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:45999)
0812 05:18:19.493310 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:19.493357 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260812 05:18:19.797019 2862 raft_consensus.cc:482] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:19.797197 2862 raft_consensus.cc:504] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:19.797485 2862 leader_election.cc:288] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
W20260812 05:18:19.797947 2436 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
W20260812 05:18:19.798028 2437 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
I20260812 05:18:19.798105 2437 leader_election.cc:302] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e95b719833149158e6b28ef0ba5955a; no voters: 02e07d591a8542f49b95ff3aef77eca7, 8729388ea95b4467a1881dd8d119cb1a
I20260812 05:18:19.798200 2862 raft_consensus.cc:2726] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:18:20.298090 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:20.297796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:44025)
0812 05:18:20.297991 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:20.298041 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260812 05:18:20.372356 2864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:20.372072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:33519)
0812 05:18:20.372217 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:20.372311 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
I20260812 05:18:20.494222 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:20.493982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:48165)
0812 05:18:20.494132 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:20.494177 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260812 05:18:21.298952 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:21.298716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:41793)
0812 05:18:21.298859 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:21.298905 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260812 05:18:21.373240 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:21.373001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:60631)
0812 05:18:21.373143 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:21.373193 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260812 05:18:21.494925 2868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:21.494702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:37647)
0812 05:18:21.494836 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:21.494875 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260812 05:18:21.954092 2869 raft_consensus.cc:482] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:21.954260 2869 raft_consensus.cc:504] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:21.954450 2869 leader_election.cc:288] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
W20260812 05:18:21.954819 2436 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
W20260812 05:18:21.954890 2437 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
I20260812 05:18:21.954963 2437 leader_election.cc:302] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e95b719833149158e6b28ef0ba5955a; no voters: 02e07d591a8542f49b95ff3aef77eca7, 8729388ea95b4467a1881dd8d119cb1a
I20260812 05:18:21.955116 2869 raft_consensus.cc:2726] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:18:22.299750 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:22.299508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:35667)
0812 05:18:22.299645 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:22.299699 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260812 05:18:22.374022 2871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:22.373795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:52937)
0812 05:18:22.373917 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:22.373957 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260812 05:18:22.495795 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:22.495535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:39745)
0812 05:18:22.495691 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:22.495745 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260812 05:18:23.300596 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:23.300350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:35553)
0812 05:18:23.300486 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:23.300545 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260812 05:18:23.374882 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:23.374577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:37847)
0812 05:18:23.374765 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:23.374838 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260812 05:18:23.496547 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:23.496285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:51105)
0812 05:18:23.496457 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:23.496501 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260812 05:18:24.301446 2877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:24.301138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:59167)
0812 05:18:24.301300 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:24.301340 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260812 05:18:24.375821 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:24.375555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:43845)
0812 05:18:24.375711 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:24.375746 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260812 05:18:24.497330 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:24.497095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:57733)
0812 05:18:24.497237 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:24.497280 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260812 05:18:25.302205 2880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:25.301921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:38415)
0812 05:18:25.302090 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:25.302136 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260812 05:18:25.376717 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:25.376352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:48019)
0812 05:18:25.376613 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:25.376656 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":91,"threads_started":1}
I20260812 05:18:25.498186 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:25.497893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:50361)
0812 05:18:25.498054 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:25.498098 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260812 05:18:26.303082 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:26.302838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:45423)
0812 05:18:26.302989 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:26.303033 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260812 05:18:26.377573 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:26.377281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:33849)
0812 05:18:26.377481 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:26.377524 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260812 05:18:26.399497 2886 raft_consensus.cc:482] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:26.399613 2886 raft_consensus.cc:504] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e07d591a8542f49b95ff3aef77eca7" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38797 } } peers { permanent_uuid: "8729388ea95b4467a1881dd8d119cb1a" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 33969 } } peers { permanent_uuid: "7e95b719833149158e6b28ef0ba5955a" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 44397 } }
I20260812 05:18:26.399796 2886 leader_election.cc:288] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797), 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969)
W20260812 05:18:26.400326 2436 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02e07d591a8542f49b95ff3aef77eca7 (127.18.166.194:38797): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
W20260812 05:18:26.400414 2437 leader_election.cc:341] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8729388ea95b4467a1881dd8d119cb1a (127.18.166.195:33969): Not found: Tablet not found: 4343316660604ebfa9656b8f060af7ba
I20260812 05:18:26.400458 2437 leader_election.cc:302] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e95b719833149158e6b28ef0ba5955a; no voters: 02e07d591a8542f49b95ff3aef77eca7, 8729388ea95b4467a1881dd8d119cb1a
I20260812 05:18:26.400532 2886 raft_consensus.cc:2726] T 4343316660604ebfa9656b8f060af7ba P 7e95b719833149158e6b28ef0ba5955a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:18:26.499009 2887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:26.498735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:57895)
0812 05:18:26.498896 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:26.498935 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260812 05:18:27.303881 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:27.303642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:52605)
0812 05:18:27.303784 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:27.303829 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260812 05:18:27.378408 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:27.378101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:38147)
0812 05:18:27.378307 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:27.378354 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260812 05:18:27.499890 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:27.499614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:37257)
0812 05:18:27.499804 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:27.499842 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":131,"threads_started":1}
I20260812 05:18:28.304725 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:28.304473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:53349)
0812 05:18:28.304626 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:28.304675 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260812 05:18:28.379285 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:28.378993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:60653)
0812 05:18:28.379182 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:28.379232 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
I20260812 05:18:28.500675 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:28.500415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:45119)
0812 05:18:28.500579 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:28.500622 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260812 05:18:29.305814 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:29.305562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:41303)
0812 05:18:29.305719 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:29.305764 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260812 05:18:29.380149 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:29.379892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:55137)
0812 05:18:29.380031 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:29.380077 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260812 05:18:29.501603 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:29.501286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:52863)
0812 05:18:29.501501 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:29.501550 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260812 05:18:30.306672 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:30.306390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:53869)
0812 05:18:30.306558 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:30.306618 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260812 05:18:30.381001 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:30.380749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:45207)
0812 05:18:30.380906 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:30.380952 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260812 05:18:30.502295 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:30.502075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:55857)
0812 05:18:30.502204 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:30.502244 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260812 05:18:31.307650 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:31.307386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.193:39441)
0812 05:18:31.307553 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:31.307600 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260812 05:18:31.382017 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:31.381762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.194:50281)
0812 05:18:31.381916 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:31.381963 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260812 05:18:31.498805 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 2383
I20260812 05:18:31.503136 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:31.502892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34219 (local address 127.18.166.195:45357)
0812 05:18:31.503027 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:31.503089 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260812 05:18:31.505048 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 2561
I20260812 05:18:31.511526 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 2703
2026-08-12T05:18:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18550 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:18:31 dist-test-slave-srgv krb5kdc[2906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:18:31 dist-test-slave-srgv krb5kdc[2906](info): set up 2 sockets
Aug 12 05:18:31 dist-test-slave-srgv krb5kdc[2906](info): commencing operation
krb5kdc: starting...
W20260812 05:18:32.587679 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:18:32 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:18:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:18:32.712205 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41555
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:32.795078 2919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:32.795300 2919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:32.795325 2919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:32.796422 2919 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 05:18:32.796458 2919 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 05:18:32.796471 2919 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 05:18:32.796483 2919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:32.796496 2919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:32.796509 2919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:32.796521 2919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:32.796540 2919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:32.798007 2919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:32.798218 2919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:32.798409 2919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:32.803406 2925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:32.803442 2927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:32.803413 2924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:32.803442 2919 server_base.cc:1034] running on GCE node
I20260812 05:18:32.803826 2919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:32.804097 2919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:32.805277 2919 hybrid_clock.cc:648] HybridClock initialized: now 1786511912805250 us; error 37 us; skew 500 ppm
Aug 12 05:18:32 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511912, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:32.807883 2919 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:32.808379 2919 webserver.cc:460] Webserver started at http://127.18.166.254:40267/ using document root <none> and password file <none>
I20260812 05:18:32.808570 2919 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:32.808619 2919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:32.808723 2919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:32.809867 2919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "6c4dd9bb5b724c08a88ddd155da047b3"
format_stamp: "Formatted at 2026-08-12 05:18:32 on dist-test-slave-srgv"
server_key: "9a32db58cbbacf6a5a75478c961dc2d2"
server_key_iv: "dc3fc2191825afb35c194230fac146c8"
server_key_version: "encryptionkey@0"
I20260812 05:18:32.810190 2919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "6c4dd9bb5b724c08a88ddd155da047b3"
format_stamp: "Formatted at 2026-08-12 05:18:32 on dist-test-slave-srgv"
server_key: "9a32db58cbbacf6a5a75478c961dc2d2"
server_key_iv: "dc3fc2191825afb35c194230fac146c8"
server_key_version: "encryptionkey@0"
I20260812 05:18:32.811528 2919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:18:32.812247 2934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:32.812426 2919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:32.812539 2919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "6c4dd9bb5b724c08a88ddd155da047b3"
format_stamp: "Formatted at 2026-08-12 05:18:32 on dist-test-slave-srgv"
server_key: "9a32db58cbbacf6a5a75478c961dc2d2"
server_key_iv: "dc3fc2191825afb35c194230fac146c8"
server_key_version: "encryptionkey@0"
I20260812 05:18:32.812633 2919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:32.828042 2919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:32.830101 2919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:32.830276 2919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:32.834746 2919 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:41555
I20260812 05:18:32.834805 2993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:41555 every 8 connection(s)
I20260812 05:18:32.835253 2919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:32.836208 2994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:32.837677 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 2919
I20260812 05:18:32.837793 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:18:32.838006 19099 external_mini_cluster.cc:1428] Setting key b018f172e190e540705f6da6bc37e8f8
I20260812 05:18:32.839135 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap starting.
I20260812 05:18:32.839888 2994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:32.840258 2994 log.cc:824] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:32.841058 2994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: No bootstrap required, opened a new log
I20260812 05:18:32.843506 2994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:32.843703 2994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:32.843741 2994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4dd9bb5b724c08a88ddd155da047b3, State: Initialized, Role: FOLLOWER
I20260812 05:18:32.843891 2994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:32.843987 2994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:32.844034 2994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:32.844118 2994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:32.844998 2994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:32.845124 2994 leader_election.cc:302] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c4dd9bb5b724c08a88ddd155da047b3; no voters:
I20260812 05:18:32.845315 2994 leader_election.cc:288] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:18:32.845381 2999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:32.845582 2999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 LEADER]: Becoming Leader. State: Replica: 6c4dd9bb5b724c08a88ddd155da047b3, State: Running, Role: LEADER
I20260812 05:18:32.845646 2994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:32.845745 2999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
Aug 12 05:18:32 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:32.846153 2999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c4dd9bb5b724c08a88ddd155da047b3. Latest consensus state: current_term: 1 leader_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } } }
I20260812 05:18:32.846155 3000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } } }
I20260812 05:18:32.846257 2999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:32.846280 3000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:32.846482 3011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:32.847199 3011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:32.849488 3011 catalog_manager.cc:1347] Generated new cluster ID: 7f0eed27872c42e7a30c485f0ab376f3
I20260812 05:18:32.849547 3011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:32.850584 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:32.838616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.18.166.254:41555)
0812 05:18:32.838802 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:32.838819 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:32.838846 (+ 27us) server_negotiation.cc:407] Connection header received
0812 05:18:32.839337 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:32.839344 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:32.839395 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:32.839519 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:32.840443 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:32.841752 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:32.842852 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:32.843284 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:32.843374 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:32.846805 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:32.846809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:32.846810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:32.846823 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:32.848831 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:32.849256 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:32.849259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:32.849259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:32.849364 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:32.849658 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:32.849659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:32.849659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:32.849961 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:32.850096 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:32.850263 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 05:18:32.850398 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260812 05:18:32.852978 3011 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:18:32.853541 3011 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:32.856376 3011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Generated new TSK 0
I20260812 05:18:32.856560 3011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:18:32.892230 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41555
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:32.995287 3024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:32.995508 3024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:32.995533 3024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:32.996726 3024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:32.996778 3024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:32.996806 3024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:32.998389 3024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:32.998637 3024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:32.998926 3024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:33.004012 3033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:33.004035 3035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:33.004077 3030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:33.004604 3024 server_base.cc:1034] running on GCE node
I20260812 05:18:33.004841 3024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:33.005115 3024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:33.006287 3024 hybrid_clock.cc:648] HybridClock initialized: now 1786511913006264 us; error 39 us; skew 500 ppm
Aug 12 05:18:33 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511913, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:33.009232 3024 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:33.009886 3024 webserver.cc:460] Webserver started at http://127.18.166.193:40779/ using document root <none> and password file <none>
I20260812 05:18:33.010214 3024 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:33.010293 3024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:33.010468 3024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:33.011844 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "86c091d5e70a4871bfb203384f5c2d62"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "b060d8599f52e6e92b1cd1b3b8284309"
server_key_iv: "4ea18c736425715b4195a8219041c0f6"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.012254 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "86c091d5e70a4871bfb203384f5c2d62"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "b060d8599f52e6e92b1cd1b3b8284309"
server_key_iv: "4ea18c736425715b4195a8219041c0f6"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.013921 3024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:18:33.014812 3043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.015033 3024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:33.015139 3024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "86c091d5e70a4871bfb203384f5c2d62"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "b060d8599f52e6e92b1cd1b3b8284309"
server_key_iv: "4ea18c736425715b4195a8219041c0f6"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.015220 3024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:33.036379 3024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:33.038925 3024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:33.039103 3024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:33.039376 3024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:33.039712 3024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:33.039769 3024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.039813 3024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:33.039842 3024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.045903 3024 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:35605
I20260812 05:18:33.045989 3170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:35605 every 8 connection(s)
I20260812 05:18:33.046376 3024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:18:33.048336 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3024
I20260812 05:18:33.048481 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:18:33.048677 19099 external_mini_cluster.cc:1428] Setting key 9a4af273b578ccc30136fb9992026923
Aug 12 05:18:33 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511913, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:33.057284 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.047224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:52949 (local address 127.18.166.254:41555)
0812 05:18:33.047308 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.047311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.047545 (+ 234us) server_negotiation.cc:407] Connection header received
0812 05:18:33.048078 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.048080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.048091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.048194 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:33.049456 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.050203 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.051247 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.051596 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.051686 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.054342 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:33.054344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:33.054344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:33.054353 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:33.055845 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.056278 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.056279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.056280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.056339 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.056590 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.056591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.056591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.056713 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:33.056785 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.057070 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.057187 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:18:33.057521 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.047167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:52949)
0812 05:18:33.047369 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.047451 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.047955 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.048254 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.048258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.049038 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:33.049321 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.049327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.050303 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.050304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.051155 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.051157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.051730 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.051731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.051848 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.052470 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:33.052480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:33.054235 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:33.055950 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:33.055958 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:33.055960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:33.056190 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:33.056413 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:33.056414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:33.056414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:33.056523 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:33.056789 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:33.056796 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:33.056934 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.057036 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.057201 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
I20260812 05:18:33.058130 3171 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41555
I20260812 05:18:33.058262 3171 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:33.058465 3171 heartbeater.cc:511] Master 127.18.166.254:41555 requested a full tablet report, sending...
I20260812 05:18:33.059034 2952 ts_manager.cc:194] Registered new tserver with Master: 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605)
I20260812 05:18:33.059654 2952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:52949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:18:33.074935 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41555
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:33.158740 3176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:33.159067 3176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:33.159111 3176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:33.160334 3176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:33.160411 3176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:33.160460 3176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:33.162020 3176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:33.162319 3176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:33.162575 3176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:33.167790 3185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:33.167776 3182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:33.167965 3176 server_base.cc:1034] running on GCE node
W20260812 05:18:33.167776 3183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:33.168309 3176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:33.168592 3176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:33.169813 3176 hybrid_clock.cc:648] HybridClock initialized: now 1786511913169795 us; error 27 us; skew 500 ppm
Aug 12 05:18:33 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511913, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:33.172621 3176 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:33.173077 3176 webserver.cc:460] Webserver started at http://127.18.166.194:33431/ using document root <none> and password file <none>
I20260812 05:18:33.173321 3176 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:33.173391 3176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:33.173581 3176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:33.174701 3176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "43a62db6817d40d4a113293a80d4e88e"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "ffc77d3824074b7ad06ec4d74182fa71"
server_key_iv: "e8dcee92aa05f7ea488e80486c3fdf0c"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.175094 3176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "43a62db6817d40d4a113293a80d4e88e"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "ffc77d3824074b7ad06ec4d74182fa71"
server_key_iv: "e8dcee92aa05f7ea488e80486c3fdf0c"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.176599 3176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:18:33.177556 3193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.177762 3176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:33.177874 3176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "43a62db6817d40d4a113293a80d4e88e"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "ffc77d3824074b7ad06ec4d74182fa71"
server_key_iv: "e8dcee92aa05f7ea488e80486c3fdf0c"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.177968 3176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:33.201977 3176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:33.204561 3176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:33.204748 3176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:33.205004 3176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:33.205358 3176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:33.205449 3176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.205516 3176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:33.205549 3176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.212077 3176 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41385
I20260812 05:18:33.212126 3314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41385 every 8 connection(s)
I20260812 05:18:33.212487 3176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:18:33 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511913, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:33.221071 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3176
I20260812 05:18:33.221215 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:18:33.221498 19099 external_mini_cluster.cc:1428] Setting key d5ed57120e2d6150fa44eefd6ba8d05b
I20260812 05:18:33.223538 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.213277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:43827 (local address 127.18.166.254:41555)
0812 05:18:33.213548 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.213565 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.213736 (+ 171us) server_negotiation.cc:407] Connection header received
0812 05:18:33.214218 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.214219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.214242 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.214297 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:33.215287 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.215872 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.216926 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.217126 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.217186 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.220021 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:33.220023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:33.220023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:33.220032 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:33.221936 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.222483 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.222485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.222486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.222555 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.222835 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.222836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.222836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.222964 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:33.223043 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.223343 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.223439 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
I20260812 05:18:33.223829 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.213258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:43827)
0812 05:18:33.213606 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.213645 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.214132 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.214352 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.214356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.214962 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:33.215168 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.215173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.215981 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.215982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.216835 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.216837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.217218 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.217219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.217339 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.217937 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:33.217946 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:33.219901 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:33.222038 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:33.222050 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:33.222053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:33.222393 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:33.222624 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:33.222625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:33.222625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:33.222763 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:33.223034 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:33.223042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:33.223198 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.223321 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.223506 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":85,"threads_started":1}
I20260812 05:18:33.224404 3315 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41555
I20260812 05:18:33.224531 3315 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:33.224730 3315 heartbeater.cc:511] Master 127.18.166.254:41555 requested a full tablet report, sending...
I20260812 05:18:33.225171 2952 ts_manager.cc:194] Registered new tserver with Master: 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
I20260812 05:18:33.225651 2952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:43827
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:18:33.248817 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41555
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:33.333056 3321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:33.333283 3321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:33.333309 3321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:33.334503 3321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:33.334554 3321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:33.334582 3321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:33.336055 3321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:33.336300 3321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:33.336555 3321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:33.341596 3329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:33.341804 3327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:33.341679 3326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:33.342097 3321 server_base.cc:1034] running on GCE node
I20260812 05:18:33.342309 3321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:33.342558 3321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:33.343739 3321 hybrid_clock.cc:648] HybridClock initialized: now 1786511913343713 us; error 38 us; skew 500 ppm
Aug 12 05:18:33 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511913, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:33.346735 3321 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:33.347323 3321 webserver.cc:460] Webserver started at http://127.18.166.195:34791/ using document root <none> and password file <none>
I20260812 05:18:33.347555 3321 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:33.347630 3321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:33.347795 3321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:18:33.348870 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "b596629beb4040458192fb57913b0620"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "58b8dc3db1da2e11610ab008aaf8b0d6"
server_key_iv: "cdbdca5f168a38730301997f3f4d21e7"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.349177 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "b596629beb4040458192fb57913b0620"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "58b8dc3db1da2e11610ab008aaf8b0d6"
server_key_iv: "cdbdca5f168a38730301997f3f4d21e7"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.350661 3321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:18:33.351506 3336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.351711 3321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:33.351787 3321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "b596629beb4040458192fb57913b0620"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "58b8dc3db1da2e11610ab008aaf8b0d6"
server_key_iv: "cdbdca5f168a38730301997f3f4d21e7"
server_key_version: "encryptionkey@0"
I20260812 05:18:33.351840 3321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:33.367003 3321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:33.368919 3321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:33.369048 3321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:33.369259 3321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:33.369644 3321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:33.369683 3321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.369706 3321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:33.369767 3321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:33.376354 3321 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:45623
I20260812 05:18:33.376441 3468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:45623 every 8 connection(s)
I20260812 05:18:33.376799 3321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:18:33 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511913, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:33.384624 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3321
I20260812 05:18:33.384738 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:18:33.384953 19099 external_mini_cluster.cc:1428] Setting key 7292f6179bf0043b4b209a2280d29afc
I20260812 05:18:33.387025 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.377657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:56479 (local address 127.18.166.254:41555)
0812 05:18:33.377711 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.377714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.377908 (+ 194us) server_negotiation.cc:407] Connection header received
0812 05:18:33.378478 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.378480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.378492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.378551 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:33.379641 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.380166 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.381242 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.381540 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.381614 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.383899 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:33.383901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:33.383901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:33.383908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:33.385618 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.386037 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.386039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.386039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.386099 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.386366 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.386366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.386367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.386489 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:33.386560 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.386849 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.386933 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:18:33.387229 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.377615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:56479)
0812 05:18:33.377770 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.377816 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.378365 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.378620 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.378624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.379333 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:33.379533 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.379538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.380267 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.380268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.381151 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.381153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.381634 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.381635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.381761 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.382335 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:33.382343 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:33.383809 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:33.385679 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:33.385693 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:33.385695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:33.385976 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:33.386171 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:33.386172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:33.386172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:33.386290 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:33.386562 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:33.386569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:33.386714 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.386827 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.386991 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260812 05:18:33.387692 3469 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41555
I20260812 05:18:33.387773 3469 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:33.387961 3469 heartbeater.cc:511] Master 127.18.166.254:41555 requested a full tablet report, sending...
I20260812 05:18:33.388501 2952 ts_manager.cc:194] Registered new tserver with Master: b596629beb4040458192fb57913b0620 (127.18.166.195:45623)
I20260812 05:18:33.388893 2952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:56479
I20260812 05:18:33.396946 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:18:33.404299 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.398430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37346 (local address 127.18.166.254:41555)
0812 05:18:33.398492 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.398494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.398628 (+ 134us) server_negotiation.cc:407] Connection header received
0812 05:18:33.398654 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.398655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.398665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.398714 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:33.399248 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.399751 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.400446 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.400649 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.400707 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.401952 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:33.401953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:33.401953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:33.401958 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:33.403365 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.403654 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.403655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.403656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.403707 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:33.403853 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:33.403854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:33.403854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:33.403985 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:33.404067 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.404158 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.404237 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:18:33.405884 2952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:18:33.407431 2952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:18:33.414238 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.409995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41385 (local address 127.0.0.1:38232)
0812 05:18:33.410371 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.410380 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.410423 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.410656 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.410657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.410659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:33.410702 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.410703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.411969 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.411970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.413071 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.413073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.413906 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.413907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.413992 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.414010 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.414086 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.414140 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260812 05:18:33.414266 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.410120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.18.166.194:41385)
0812 05:18:33.410362 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.410367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.410418 (+ 51us) server_negotiation.cc:407] Connection header received
0812 05:18:33.410488 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.410490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.410507 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.410582 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:33.410782 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.411856 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.413155 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.413816 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.413898 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.413951 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.414116 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.414157 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":91,"threads_started":1}
I20260812 05:18:33.414851 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.409790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45623 (local address 127.0.0.1:33596)
0812 05:18:33.410139 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.410185 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.410262 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.410787 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.410788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.410790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:33.410837 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.410838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.411713 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.411714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.413070 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.413073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.414594 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.414595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.414663 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.414668 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.414739 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.414782 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":6,"thread_start_us":68,"threads_started":1}
I20260812 05:18:33.414897 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.409908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.18.166.195:45623)
0812 05:18:33.410199 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.410204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.410241 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:18:33.410354 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.410356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.410372 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.410450 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:33.410916 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.411607 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.413503 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.414250 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.414505 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.414567 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.414764 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.414814 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":96,"threads_started":1}
I20260812 05:18:33.415235 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.409790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35605 (local address 127.0.0.1:58528)
0812 05:18:33.410139 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.410185 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.410265 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.410515 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.410518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.410520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:33.410594 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.410597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.413178 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.413179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.413945 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.413946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.414944 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.414945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.415013 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.415017 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.415071 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.415118 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"spinlock_wait_cycles":768,"thread_start_us":85,"threads_started":1}
I20260812 05:18:33.415159 3245 tablet_service.cc:1467] Processing CreateTablet for tablet d10294c6b46340d6b3065ac4de6277c5 (DEFAULT_TABLE table=test-table [id=c9ed82ac0a234f938542abfa8d792c53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:33.415477 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.409909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.18.166.193:35605)
0812 05:18:33.410117 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.410122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.410379 (+ 257us) server_negotiation.cc:407] Connection header received
0812 05:18:33.410424 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.410427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.410441 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.410516 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:33.410685 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.411857 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.414042 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.414788 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.414857 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.414906 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.415357 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.415403 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260812 05:18:33.415521 3245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10294c6b46340d6b3065ac4de6277c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:33.415685 3392 tablet_service.cc:1467] Processing CreateTablet for tablet d10294c6b46340d6b3065ac4de6277c5 (DEFAULT_TABLE table=test-table [id=c9ed82ac0a234f938542abfa8d792c53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:33.415931 3392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10294c6b46340d6b3065ac4de6277c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:33.418280 3487 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Bootstrap starting.
I20260812 05:18:33.418817 3488 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Bootstrap starting.
I20260812 05:18:33.418785 3098 tablet_service.cc:1467] Processing CreateTablet for tablet d10294c6b46340d6b3065ac4de6277c5 (DEFAULT_TABLE table=test-table [id=c9ed82ac0a234f938542abfa8d792c53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:33.419091 3487 tablet_bootstrap.cc:654] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:33.419164 3098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10294c6b46340d6b3065ac4de6277c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:33.419486 3488 tablet_bootstrap.cc:654] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:33.419651 3487 log.cc:824] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:33.420063 3488 log.cc:824] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:33.420418 3487 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: No bootstrap required, opened a new log
I20260812 05:18:33.420498 3487 ts_tablet_manager.cc:1397] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:33.420863 3488 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: No bootstrap required, opened a new log
I20260812 05:18:33.420956 3488 ts_tablet_manager.cc:1397] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:33.422230 3487 raft_consensus.cc:348] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.422369 3487 raft_consensus.cc:374] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:33.422402 3487 raft_consensus.cc:729] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a62db6817d40d4a113293a80d4e88e, State: Initialized, Role: FOLLOWER
I20260812 05:18:33.422490 3487 consensus_queue.cc:260] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.422674 3491 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap starting.
I20260812 05:18:33.422782 3487 ts_tablet_manager.cc:1428] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:18:33.422901 3315 heartbeater.cc:503] Master 127.18.166.254:41555 was elected leader, sending a full tablet report...
I20260812 05:18:33.423352 3491 tablet_bootstrap.cc:654] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:33.423339 3488 raft_consensus.cc:348] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.423507 3488 raft_consensus.cc:374] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:33.423549 3488 raft_consensus.cc:729] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b596629beb4040458192fb57913b0620, State: Initialized, Role: FOLLOWER
I20260812 05:18:33.423664 3488 consensus_queue.cc:260] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.423934 3491 log.cc:824] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:33.423969 3488 ts_tablet_manager.cc:1428] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:18:33.424098 3469 heartbeater.cc:503] Master 127.18.166.254:41555 was elected leader, sending a full tablet report...
I20260812 05:18:33.424983 3491 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: No bootstrap required, opened a new log
I20260812 05:18:33.425158 3491 ts_tablet_manager.cc:1397] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:18:33.426630 3491 raft_consensus.cc:348] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.426771 3491 raft_consensus.cc:374] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:33.426795 3491 raft_consensus.cc:729] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c091d5e70a4871bfb203384f5c2d62, State: Initialized, Role: FOLLOWER
I20260812 05:18:33.426908 3491 consensus_queue.cc:260] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.427203 3491 ts_tablet_manager.cc:1428] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:18:33.427238 3171 heartbeater.cc:503] Master 127.18.166.254:41555 was elected leader, sending a full tablet report...
W20260812 05:18:33.463109 3316 tablet.cc:2369] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:33.463351 3496 raft_consensus.cc:482] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:33.463502 3496 raft_consensus.cc:504] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.463874 3496 leader_election.cc:288] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
I20260812 05:18:33.466794 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.464098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45623 (local address 127.18.166.193:59921)
0812 05:18:33.464160 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.464173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.464219 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.464403 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.464405 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.464407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:33.464486 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.464488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.465120 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.465121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.465946 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.465949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.466522 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.466522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.466606 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.466609 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.466678 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.466726 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:18:33.466794 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.464203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:59921 (local address 127.18.166.195:45623)
0812 05:18:33.464278 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.464280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.464290 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:18:33.464319 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.464320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.464329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.464396 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:33.464563 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.465035 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.466043 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.466436 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.466524 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.466533 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.466685 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.466725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:18:33.467061 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.464338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:45445 (local address 127.18.166.194:41385)
0812 05:18:33.464395 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.464397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.464522 (+ 125us) server_negotiation.cc:407] Connection header received
0812 05:18:33.464595 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.464600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.464611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.464671 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:33.464840 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.465292 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.466253 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.466624 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.466724 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.466732 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.466954 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.467001 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:18:33.467068 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.464223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41385 (local address 127.18.166.193:45445)
0812 05:18:33.464443 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.464452 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:33.464530 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:33.464675 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:33.464675 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:33.464677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:33.464759 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.464760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.465386 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.465388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.466153 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:33.466155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.466725 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.466726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:33.466824 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.466828 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:33.466937 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 05:18:33.466992 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":132,"threads_started":1}
I20260812 05:18:33.467139 3417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10294c6b46340d6b3065ac4de6277c5" candidate_uuid: "86c091d5e70a4871bfb203384f5c2d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b596629beb4040458192fb57913b0620" is_pre_election: true
I20260812 05:18:33.467285 3417 raft_consensus.cc:2454] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c091d5e70a4871bfb203384f5c2d62 in term 0.
I20260812 05:18:33.467563 3045 leader_election.cc:302] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c091d5e70a4871bfb203384f5c2d62, b596629beb4040458192fb57913b0620; no voters:
I20260812 05:18:33.467703 3496 raft_consensus.cc:2781] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:18:33.467762 3496 raft_consensus.cc:482] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:18:33.467782 3496 raft_consensus.cc:3037] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:33.467705 3266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10294c6b46340d6b3065ac4de6277c5" candidate_uuid: "86c091d5e70a4871bfb203384f5c2d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a62db6817d40d4a113293a80d4e88e" is_pre_election: true
I20260812 05:18:33.467883 3266 raft_consensus.cc:2454] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c091d5e70a4871bfb203384f5c2d62 in term 0.
I20260812 05:18:33.468633 3496 raft_consensus.cc:504] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.468772 3496 leader_election.cc:288] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 1 election: Requested vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
I20260812 05:18:33.468937 3266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10294c6b46340d6b3065ac4de6277c5" candidate_uuid: "86c091d5e70a4871bfb203384f5c2d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a62db6817d40d4a113293a80d4e88e"
I20260812 05:18:33.468971 3417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10294c6b46340d6b3065ac4de6277c5" candidate_uuid: "86c091d5e70a4871bfb203384f5c2d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b596629beb4040458192fb57913b0620"
I20260812 05:18:33.469038 3266 raft_consensus.cc:3037] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:33.469039 3417 raft_consensus.cc:3037] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:33.469837 3266 raft_consensus.cc:2454] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c091d5e70a4871bfb203384f5c2d62 in term 1.
I20260812 05:18:33.469866 3417 raft_consensus.cc:2454] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c091d5e70a4871bfb203384f5c2d62 in term 1.
I20260812 05:18:33.470044 3045 leader_election.cc:302] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a62db6817d40d4a113293a80d4e88e, 86c091d5e70a4871bfb203384f5c2d62; no voters:
I20260812 05:18:33.470151 3496 raft_consensus.cc:2781] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:33.470319 3496 raft_consensus.cc:686] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 LEADER]: Becoming Leader. State: Replica: 86c091d5e70a4871bfb203384f5c2d62, State: Running, Role: LEADER
I20260812 05:18:33.470418 3496 consensus_queue.cc:237] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:33.471330 2951 catalog_manager.cc:5495] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193). New cstate: current_term: 1 leader_uuid: "86c091d5e70a4871bfb203384f5c2d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } health_report { overall_health: UNKNOWN } } }
I20260812 05:18:33.490361 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.488104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.18.166.193:35605)
0812 05:18:33.488175 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:33.488177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:33.488185 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:18:33.488235 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:33.488236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:33.488254 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:33.488314 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:33.488440 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.488898 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.489709 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:33.489895 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:33.489949 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:33.490077 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:33.490173 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:18:33.490239 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:33.490269 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 05:18:33.490286 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:18:33.492167 3266 raft_consensus.cc:1261] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Refusing update from remote peer 86c091d5e70a4871bfb203384f5c2d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:33.492197 3417 raft_consensus.cc:1261] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Refusing update from remote peer 86c091d5e70a4871bfb203384f5c2d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:33.492552 3496 consensus_queue.cc:1035] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:33.492713 3499 consensus_queue.cc:1035] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:33.495653 3503 mvcc.cc:205] Tried to move back new op lower bound from 7317552797661130752 to 7317552797575450624. Current Snapshot: MvccSnapshot[applied={T|T < 7317552797661130752 or (T in {7317552797661130752})}]
I20260812 05:18:33.497124 3504 mvcc.cc:205] Tried to move back new op lower bound from 7317552797661130752 to 7317552797575450624. Current Snapshot: MvccSnapshot[applied={T|T < 7317552797661130752 or (T in {7317552797661130752})}]
I20260812 05:18:33.497318 2952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:33.497398 2952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:33.498231 2952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Sending DeleteTablet for 3 replicas of tablet d10294c6b46340d6b3065ac4de6277c5
I20260812 05:18:33.498580 3392 tablet_service.cc:1514] Processing DeleteTablet for tablet d10294c6b46340d6b3065ac4de6277c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:33 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:33596
I20260812 05:18:33.498592 3098 tablet_service.cc:1514] Processing DeleteTablet for tablet d10294c6b46340d6b3065ac4de6277c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:33 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58528
I20260812 05:18:33.498637 3245 tablet_service.cc:1514] Processing DeleteTablet for tablet d10294c6b46340d6b3065ac4de6277c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:33 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:38232
I20260812 05:18:33.498842 3515 tablet_replica.cc:330] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: stopping tablet replica
I20260812 05:18:33.498881 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3024
I20260812 05:18:33.498945 3515 raft_consensus.cc:2229] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:33.499053 3515 raft_consensus.cc:2258] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:33.500931 3515 ts_tablet_manager.cc:1905] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:33.503144 3515 ts_tablet_manager.cc:1918] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:33.503227 3515 log.cc:1196] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/d10294c6b46340d6b3065ac4de6277c5
I20260812 05:18:33.503582 3515 ts_tablet_manager.cc:1939] T d10294c6b46340d6b3065ac4de6277c5 P 43a62db6817d40d4a113293a80d4e88e: Deleting consensus metadata
I20260812 05:18:33.504107 2936 catalog_manager.cc:4841] TS 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): tablet d10294c6b46340d6b3065ac4de6277c5 (table test-table [id=c9ed82ac0a234f938542abfa8d792c53]) successfully deleted
I20260812 05:18:33.505955 3513 tablet_replica.cc:330] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: stopping tablet replica
I20260812 05:18:33.506096 3513 raft_consensus.cc:2229] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:33.506238 3513 raft_consensus.cc:2258] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:33.506873 3513 ts_tablet_manager.cc:1905] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:33.508776 3513 ts_tablet_manager.cc:1918] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:33.508859 3513 log.cc:1196] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/d10294c6b46340d6b3065ac4de6277c5
W20260812 05:18:33.508944 2937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:35605 (error 108)
I20260812 05:18:33.509148 3513 ts_tablet_manager.cc:1939] T d10294c6b46340d6b3065ac4de6277c5 P b596629beb4040458192fb57913b0620: Deleting consensus metadata
I20260812 05:18:33.509222 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3176
I20260812 05:18:33.511158 2936 catalog_manager.cc:4841] TS b596629beb4040458192fb57913b0620 (127.18.166.195:45623): tablet d10294c6b46340d6b3065ac4de6277c5 (table test-table [id=c9ed82ac0a234f938542abfa8d792c53]) successfully deleted
I20260812 05:18:33.513898 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:33.513767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35605 (local address 127.0.0.1:58546)
0812 05:18:33.513828 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:33.513869 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 05:18:33.514045 2937 catalog_manager.cc:4584] TS 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d10294c6b46340d6b3065ac4de6277c5: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35605: connect: Connection refused (error 111)
I20260812 05:18:33.514927 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3321
I20260812 05:18:33.526007 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 2919
I20260812 05:18:53.532737 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41555
--webserver_interface=127.18.166.254
--webserver_port=40267
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:18:53.615465 3517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:53.615721 3517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:53.615759 3517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:53.616886 3517 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 05:18:53.616957 3517 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 05:18:53.616979 3517 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 05:18:53.616997 3517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:18:53.617030 3517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:53.617061 3517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:53.617091 3517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:18:53.617120 3517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:18:53.618602 3517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=40267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:53.618897 3517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:53.619138 3517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:53.628270 3523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:53.628293 3524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:53.628281 3526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:53.628353 3517 server_base.cc:1034] running on GCE node
I20260812 05:18:53.628813 3517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:53.629124 3517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:53.630306 3517 hybrid_clock.cc:648] HybridClock initialized: now 1786511933630295 us; error 49 us; skew 500 ppm
Aug 12 05:18:53 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511933, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:53.632833 3517 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:18:53.633322 3517 webserver.cc:460] Webserver started at http://127.18.166.254:40267/ using document root <none> and password file <none>
I20260812 05:18:53.633587 3517 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:53.633682 3517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:53.635169 3517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:18:53.635813 3534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:53.635993 3517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:18:53.636102 3517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "6c4dd9bb5b724c08a88ddd155da047b3"
format_stamp: "Formatted at 2026-08-12 05:18:32 on dist-test-slave-srgv"
server_key: "9a32db58cbbacf6a5a75478c961dc2d2"
server_key_iv: "dc3fc2191825afb35c194230fac146c8"
server_key_version: "encryptionkey@0"
I20260812 05:18:53.636528 3517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:53.653334 3517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:53.655364 3517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:53.655541 3517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:53.660040 3517 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:41555
I20260812 05:18:53.660096 3589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:41555 every 8 connection(s)
I20260812 05:18:53.660531 3517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:18:53.662144 3590 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:18:53.662781 3590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap starting.
I20260812 05:18:53.668749 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3517
I20260812 05:18:53.669121 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:35605
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:41555
--webserver_port=40779
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:53.672573 3590 log.cc:824] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:53.674717 3590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:53.675048 3590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap complete.
I20260812 05:18:53.677475 3590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:53.677718 3590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4dd9bb5b724c08a88ddd155da047b3, State: Initialized, Role: FOLLOWER
I20260812 05:18:53.677876 3590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:53.677943 3590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:18:53.678006 3590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:18:53.678059 3590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:18:53.679122 3590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:53.679208 3590 leader_election.cc:302] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c4dd9bb5b724c08a88ddd155da047b3; no voters:
I20260812 05:18:53.679415 3590 leader_election.cc:288] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:18:53.679452 3595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:18:53.679576 3595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 LEADER]: Becoming Leader. State: Replica: 6c4dd9bb5b724c08a88ddd155da047b3, State: Running, Role: LEADER
I20260812 05:18:53.679663 3595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:18:53.679692 3590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:18:53.680338 3596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } } }
I20260812 05:18:53.680343 3597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c4dd9bb5b724c08a88ddd155da047b3. Latest consensus state: current_term: 2 leader_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } } }
I20260812 05:18:53.680441 3596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:53.680445 3597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:18:53.680779 3605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:18:53.681666 3605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:18:53.681794 3605 catalog_manager.cc:1259] Loaded cluster ID: 7f0eed27872c42e7a30c485f0ab376f3
I20260812 05:18:53.682053 3605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:18:53.682847 3605 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:18:53.683037 3605 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Loaded TSK: 0
I20260812 05:18:53.689098 3605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Generated new TSK 1
I20260812 05:18:53.689209 3605 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:18:53.764740 3593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:53.764940 3593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:53.764962 3593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:53.766137 3593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:53.766181 3593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:53.766208 3593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:18:53.767648 3593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:35605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=40779
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:53.767877 3593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:53.768123 3593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:53.773046 3619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:53.773058 3620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:53.773033 3622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:53.773294 3593 server_base.cc:1034] running on GCE node
I20260812 05:18:53.773546 3593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:53.773797 3593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:53.774960 3593 hybrid_clock.cc:648] HybridClock initialized: now 1786511933774947 us; error 33 us; skew 500 ppm
Aug 12 05:18:53 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511933, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:53.777534 3593 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:18:53.778038 3593 webserver.cc:460] Webserver started at http://127.18.166.193:40779/ using document root <none> and password file <none>
I20260812 05:18:53.778273 3593 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:53.778357 3593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:53.779767 3593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:53.780613 3629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:53.780777 3593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:53.780887 3593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "86c091d5e70a4871bfb203384f5c2d62"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "b060d8599f52e6e92b1cd1b3b8284309"
server_key_iv: "4ea18c736425715b4195a8219041c0f6"
server_key_version: "encryptionkey@0"
I20260812 05:18:53.781304 3593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:53.807571 3593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:53.809887 3593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:53.810038 3593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:53.810338 3593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:53.810844 3638 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:18:53.812682 3593 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:18:53.812752 3593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 05:18:53.812801 3593 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:18:53.813522 3593 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:18:53.813580 3593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:53.813643 3638 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap starting.
I20260812 05:18:53.821089 3593 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:35605
I20260812 05:18:53.821237 3759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:35605 every 8 connection(s)
I20260812 05:18:53.821597 3593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:18:53.824420 3638 log.cc:824] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:53.825429 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3593
I20260812 05:18:53.825745 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:41385
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:41555
--webserver_port=33431
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:18:53.827559 3638 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:18:53.827859 3638 tablet_bootstrap.cc:492] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap complete.
I20260812 05:18:53.829062 3638 ts_tablet_manager.cc:1397] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260812 05:18:53.830329 3638 raft_consensus.cc:348] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:53.830736 3638 raft_consensus.cc:729] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c091d5e70a4871bfb203384f5c2d62, State: Initialized, Role: FOLLOWER
I20260812 05:18:53.830946 3638 consensus_queue.cc:260] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } } peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } }
I20260812 05:18:53.831262 3638 ts_tablet_manager.cc:1428] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Aug 12 05:18:53 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511933, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:53.835960 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:53.822435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:38567 (local address 127.18.166.254:41555)
0812 05:18:53.824623 (+ 2188us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:53.824652 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:53.824700 (+ 48us) server_negotiation.cc:407] Connection header received
0812 05:18:53.824843 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:53.824849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:53.824921 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:53.825070 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:53.825964 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.826864 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.828440 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.829103 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.829233 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:53.832181 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:53.832185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:53.832185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:53.832196 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:53.833985 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:53.834528 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:53.834530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:53.834530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:53.834614 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:53.834989 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:53.834990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:53.834990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:53.835266 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:53.835334 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:53.835709 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 05:18:53.835821 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2141,"thread_start_us":96,"threads_started":1}
I20260812 05:18:53.836130 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:53.822344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:38567)
0812 05:18:53.822544 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:53.822589 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:53.823054 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:53.825096 (+ 2042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:53.825100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:53.825538 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:53.825785 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:53.825791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.826997 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.826999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:53.828329 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:53.828333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.829255 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.829257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:53.829433 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:53.830081 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:53.830092 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:53.832067 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:53.834072 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:53.834081 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:53.834083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:53.834433 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:53.834687 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:53.834688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:53.834689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:53.834858 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:53.835342 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:53.835350 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:53.835531 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 05:18:53.835665 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 05:18:53.835877 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260812 05:18:53.836910 3760 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41555
I20260812 05:18:53.837018 3760 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:53.837212 3760 heartbeater.cc:511] Master 127.18.166.254:41555 requested a full tablet report, sending...
I20260812 05:18:53.838096 3551 ts_manager.cc:194] Registered new tserver with Master: 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605)
I20260812 05:18:53.839490 3551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:38567
I20260812 05:18:53.840413 3760 heartbeater.cc:503] Master 127.18.166.254:41555 was elected leader, sending a full tablet report...
Aug 12 05:18:53 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511933, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.193@KRBTEST.COM
I20260812 05:18:53.847796 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:53.838791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35605 (local address 127.0.0.1:35176)
0812 05:18:53.839010 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:53.839047 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:53.839115 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:53.839437 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:53.839443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:53.839827 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:53.839890 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:53.839892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.841551 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.841552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:53.842127 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:53.842128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.842605 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.842606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:53.842684 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:53.843211 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:53.843216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:53.844746 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:53.846755 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:53.846763 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:53.846765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:53.846958 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:53.847169 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:53.847169 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:53.847170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:53.847222 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:53.847520 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:53.847526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:53.847569 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 05:18:53.847631 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:18:53.847725 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":158,"threads_started":1}
I20260812 05:18:53.847904 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:53.839071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.18.166.193:35605)
0812 05:18:53.839244 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:53.839248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:53.839297 (+ 49us) server_negotiation.cc:407] Connection header received
0812 05:18:53.839331 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:53.839333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:53.839345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:53.839434 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:53.840559 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.841468 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.842226 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.842527 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.842604 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:53.844843 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:53.844852 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:53.844853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:53.844862 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:53.846659 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:53.847033 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:53.847037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:53.847037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:53.847102 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:53.847280 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:53.847281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:53.847281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:53.847458 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:53.847539 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:53.847687 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 05:18:53.847806 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 05:18:53.848306 3688 tablet_service.cc:1514] Processing DeleteTablet for tablet d10294c6b46340d6b3065ac4de6277c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:33 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:35176
I20260812 05:18:53.848929 3770 tablet_replica.cc:330] stopping tablet replica
I20260812 05:18:53.849061 3770 raft_consensus.cc:2229] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:53.849154 3770 raft_consensus.cc:2258] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:53.849540 3770 ts_tablet_manager.cc:1905] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:53.852123 3770 ts_tablet_manager.cc:1918] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:53.852219 3770 log.cc:1196] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal/wals/d10294c6b46340d6b3065ac4de6277c5
I20260812 05:18:53.852463 3770 ts_tablet_manager.cc:1939] T d10294c6b46340d6b3065ac4de6277c5 P 86c091d5e70a4871bfb203384f5c2d62: Deleting consensus metadata
I20260812 05:18:53.853000 3537 catalog_manager.cc:4841] TS 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605): tablet d10294c6b46340d6b3065ac4de6277c5 (table test-table [id=c9ed82ac0a234f938542abfa8d792c53]) successfully deleted
W20260812 05:18:53.915719 3765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:53.915928 3765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:53.915952 3765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:53.917040 3765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:53.917083 3765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:53.917109 3765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:18:53.918587 3765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:41385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=33431
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:53.918788 3765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:53.918973 3765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:53.924165 3775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:53.924156 3776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:53.924156 3779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:53.924660 3765 server_base.cc:1034] running on GCE node
I20260812 05:18:53.924883 3765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:53.925133 3765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:53.926306 3765 hybrid_clock.cc:648] HybridClock initialized: now 1786511933926280 us; error 35 us; skew 500 ppm
Aug 12 05:18:53 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511933, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:53.928835 3765 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:18:53.929322 3765 webserver.cc:460] Webserver started at http://127.18.166.194:33431/ using document root <none> and password file <none>
I20260812 05:18:53.929601 3765 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:53.929672 3765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:53.931113 3765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:53.931680 3787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:53.931875 3765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:18:53.931947 3765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "43a62db6817d40d4a113293a80d4e88e"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "ffc77d3824074b7ad06ec4d74182fa71"
server_key_iv: "e8dcee92aa05f7ea488e80486c3fdf0c"
server_key_version: "encryptionkey@0"
I20260812 05:18:53.932284 3765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:53.947517 3765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:53.951308 3765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:53.951493 3765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:53.951781 3765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:53.952135 3765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:53.952195 3765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:53.952237 3765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:53.952258 3765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:53.958545 3765 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41385
I20260812 05:18:53.958630 3907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41385 every 8 connection(s)
I20260812 05:18:53.958981 3765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:18:53.962921 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3765
I20260812 05:18:53.963201 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:45623
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:41555
--webserver_port=34791
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:18:53 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511933, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:53.969554 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:53.959778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:45273 (local address 127.18.166.254:41555)
0812 05:18:53.959845 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:53.959847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:53.960074 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:18:53.960626 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:53.960627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:53.960644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:53.960709 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:53.961752 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.962271 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.963397 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.963647 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.963728 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:53.966197 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:53.966199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:53.966199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:53.966208 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:53.968020 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:53.968437 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:53.968439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:53.968439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:53.968510 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:53.968752 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:53.968753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:53.968753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:53.968927 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:53.969009 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:53.969259 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 05:18:53.969371 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:18:53.969651 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:53.959765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:45273)
0812 05:18:53.959949 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:53.959988 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:53.960509 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:53.960778 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:53.960783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:53.961383 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:53.961646 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:53.961651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.962368 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.962369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:53.963239 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:53.963242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:53.963735 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:53.963737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:53.963867 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:53.964420 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:53.964429 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:53.966091 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:53.968107 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:53.968115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:53.968117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:53.968357 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:53.968582 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:53.968582 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:53.968583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:53.968685 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:53.969000 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:53.969005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:53.969124 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 05:18:53.969208 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:18:53.969350 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260812 05:18:53.970311 3908 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41555
I20260812 05:18:53.970409 3908 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:53.970607 3908 heartbeater.cc:511] Master 127.18.166.254:41555 requested a full tablet report, sending...
I20260812 05:18:53.971038 3551 ts_manager.cc:194] Registered new tserver with Master: 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
I20260812 05:18:53.971419 3551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:45273
W20260812 05:18:54.047734 3911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:18:54.047995 3911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:18:54.048038 3911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:18:54.049247 3911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:18:54.049331 3911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:18:54.049378 3911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:18:54.050930 3911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:45623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=34791
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:18:54.051214 3911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:18:54.051468 3911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:18:54.056679 3920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:54.056685 3922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:18:54.056703 3919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:18:54.056933 3911 server_base.cc:1034] running on GCE node
I20260812 05:18:54.057169 3911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:18:54.057518 3911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:18:54.058705 3911 hybrid_clock.cc:648] HybridClock initialized: now 1786511934058688 us; error 47 us; skew 500 ppm
Aug 12 05:18:54 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511934, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:18:54.061507 3911 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:18:54.061957 3911 webserver.cc:460] Webserver started at http://127.18.166.195:34791/ using document root <none> and password file <none>
I20260812 05:18:54.062172 3911 fs_manager.cc:359] Metadata directory not provided
I20260812 05:18:54.062235 3911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:18:54.063591 3911 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:54.064186 3930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:54.064378 3911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:54.064471 3911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "b596629beb4040458192fb57913b0620"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "58b8dc3db1da2e11610ab008aaf8b0d6"
server_key_iv: "cdbdca5f168a38730301997f3f4d21e7"
server_key_version: "encryptionkey@0"
I20260812 05:18:54.064813 3911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:18:54.080688 3911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:18:54.083014 3911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:18:54.083177 3911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:18:54.083508 3911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:18:54.083841 3911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:18:54.083874 3911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:54.083930 3911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:18:54.083959 3911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:18:54.090080 3911 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:45623
I20260812 05:18:54.090173 4061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:45623 every 8 connection(s)
I20260812 05:18:54.090528 3911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:18:54 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511934, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:18:54.099460 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 3911
I20260812 05:18:54.100550 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.091293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:51115 (local address 127.18.166.254:41555)
0812 05:18:54.091366 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.091373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.091585 (+ 212us) server_negotiation.cc:407] Connection header received
0812 05:18:54.092135 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.092136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.092150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.092219 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:54.093238 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.093815 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.094838 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.095051 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.095130 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.097510 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:54.097512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:54.097512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:54.097521 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:54.098959 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:54.099390 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:54.099391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:54.099391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:54.099457 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:54.099791 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:54.099792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:54.099792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:54.099963 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:54.100053 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.100362 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.100460 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:18:54.100739 4064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.091296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:51115)
0812 05:18:54.091459 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:54.091496 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:54.092020 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:54.092309 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:54.092312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:54.092921 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:18:54.093124 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.093129 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.093918 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.093919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.094733 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.094735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.095139 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.095141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.095283 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.095864 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:18:54.095873 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:18:54.097351 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:18:54.099048 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:54.099057 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:54.099059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:54.099308 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:54.099530 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:18:54.099531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:18:54.099531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:18:54.099708 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:18:54.100043 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:18:54.100051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:18:54.100204 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 05:18:54.100325 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 05:18:54.100500 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260812 05:18:54.101255 4062 heartbeater.cc:347] Connected to a master server at 127.18.166.254:41555
I20260812 05:18:54.101374 4062 heartbeater.cc:464] Registering TS with master...
I20260812 05:18:54.101603 4062 heartbeater.cc:511] Master 127.18.166.254:41555 requested a full tablet report, sending...
I20260812 05:18:54.102063 3551 ts_manager.cc:194] Registered new tserver with Master: b596629beb4040458192fb57913b0620 (127.18.166.195:45623)
I20260812 05:18:54.102504 3551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:51115
I20260812 05:18:54.106950 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.099937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.18.166.254:41555)
0812 05:18:54.100221 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.100223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.100229 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:18:54.100246 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.100247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.100256 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.100322 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:54.101174 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.101865 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.102785 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.102981 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.103057 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.104335 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:54.104336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:54.104336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:54.104436 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:54.105777 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:54.106132 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:54.106133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:54.106134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:54.106197 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:54.106414 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:54.106415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:54.106415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:54.106529 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:54.106726 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.106764 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.106847 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":79,"threads_started":1}
I20260812 05:18:54.107779 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:18:54.110378 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.108141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.18.166.254:41555)
0812 05:18:54.108182 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.108184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.108346 (+ 162us) server_negotiation.cc:407] Connection header received
0812 05:18:54.108392 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.108392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.108421 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.108480 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:54.108574 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.109024 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.109795 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.109976 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.110027 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.110155 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:54.110198 (+ 43us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 05:18:54.110315 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47820: token expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:18:54.110442 4067 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47820: token expired
W20260812 05:18:54.110600 3473 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.166.254:41555)
I20260812 05:18:54.115989 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.110868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.18.166.254:41555)
0812 05:18:54.110919 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.110921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.110927 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:18:54.110942 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.110943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.110951 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.110993 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:18:54.111429 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.111853 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.112479 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.112641 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.112686 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.113881 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:18:54.113882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:18:54.113882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:18:54.113887 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:18:54.115097 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:54.115439 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:54.115439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:54.115440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:54.115489 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:18:54.115632 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:18:54.115633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:18:54.115633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:18:54.115721 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:18:54.115802 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.115852 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.115907 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:18:54.117148 3551 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:18:54.118801 3551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:18:54.121623 3688 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf01aad5b0416697afd283fddad681 (DEFAULT_TABLE table=test-table [id=f94f60b1ed3b48f0bdf10af029fc0cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:54.121850 3688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf01aad5b0416697afd283fddad681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:54.123018 3638 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap starting.
I20260812 05:18:54.123333 3638 tablet_bootstrap.cc:654] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:54.124197 3638 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: No bootstrap required, opened a new log
I20260812 05:18:54.124261 3638 ts_tablet_manager.cc:1397] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:18:54.124423 3638 raft_consensus.cc:348] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.124501 3638 raft_consensus.cc:374] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:54.124559 3638 raft_consensus.cc:729] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c091d5e70a4871bfb203384f5c2d62, State: Initialized, Role: FOLLOWER
I20260812 05:18:54.124603 3638 consensus_queue.cc:260] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.124686 3638 ts_tablet_manager.cc:1428] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:18:54.124915 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.120788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45623 (local address 127.0.0.1:34588)
0812 05:18:54.120823 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:54.120861 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:54.120901 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:54.121518 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:54.121519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:54.121521 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:54.121570 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.121572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.122642 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.122643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.123711 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.123715 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.124652 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.124653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.124744 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.124768 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 05:18:54.124796 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 05:18:54.124840 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 05:18:54.125319 4070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.121125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.18.166.195:45623)
0812 05:18:54.121316 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.121321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.121331 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:18:54.121377 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.121379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.121392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.121513 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:54.121658 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.122545 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.123805 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.124553 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.125089 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.125160 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.125185 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.125226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260812 05:18:54.126091 3987 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf01aad5b0416697afd283fddad681 (DEFAULT_TABLE table=test-table [id=f94f60b1ed3b48f0bdf10af029fc0cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:54.126441 3987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf01aad5b0416697afd283fddad681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:54.127276 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.120890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41385 (local address 127.0.0.1:42678)
0812 05:18:54.121104 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:54.121143 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:54.121225 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:54.123554 (+ 2329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:54.123555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:54.123557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:54.123603 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.123604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.125073 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.125074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.126192 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.126196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.126980 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.126982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.127051 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.127055 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:54.127124 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:18:54.127164 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":119,"threads_started":1}
I20260812 05:18:54.127295 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.123067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.18.166.194:41385)
0812 05:18:54.123287 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.123297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.123306 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:18:54.123339 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.123340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.123352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.123451 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:54.123679 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.124979 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.126307 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.126848 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.126909 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.126940 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.127145 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.127198 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":108,"threads_started":1}
I20260812 05:18:54.129314 4073 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Bootstrap starting.
I20260812 05:18:54.129854 3837 tablet_service.cc:1467] Processing CreateTablet for tablet 16bf01aad5b0416697afd283fddad681 (DEFAULT_TABLE table=test-table [id=f94f60b1ed3b48f0bdf10af029fc0cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:18:54.130156 4073 tablet_bootstrap.cc:654] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:54.130254 3837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bf01aad5b0416697afd283fddad681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:18:54.130717 4073 log.cc:824] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:54.131548 4073 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: No bootstrap required, opened a new log
I20260812 05:18:54.131603 4073 ts_tablet_manager.cc:1397] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:54.133095 4075 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Bootstrap starting.
I20260812 05:18:54.133018 4073 raft_consensus.cc:348] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.133203 4073 raft_consensus.cc:374] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:54.133251 4073 raft_consensus.cc:729] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b596629beb4040458192fb57913b0620, State: Initialized, Role: FOLLOWER
I20260812 05:18:54.133374 4073 consensus_queue.cc:260] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.133740 4062 heartbeater.cc:503] Master 127.18.166.254:41555 was elected leader, sending a full tablet report...
I20260812 05:18:54.133754 4073 ts_tablet_manager.cc:1428] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:54.133970 4075 tablet_bootstrap.cc:654] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Neither blocks nor log segments found. Creating new log.
I20260812 05:18:54.134578 4075 log.cc:824] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Log is configured to *not* fsync() on all Append() calls
I20260812 05:18:54.135383 4075 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: No bootstrap required, opened a new log
I20260812 05:18:54.135437 4075 ts_tablet_manager.cc:1397] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:18:54.136965 4075 raft_consensus.cc:348] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.137096 4075 raft_consensus.cc:374] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:18:54.137118 4075 raft_consensus.cc:729] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a62db6817d40d4a113293a80d4e88e, State: Initialized, Role: FOLLOWER
I20260812 05:18:54.137269 4075 consensus_queue.cc:260] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.137612 3908 heartbeater.cc:503] Master 127.18.166.254:41555 was elected leader, sending a full tablet report...
I20260812 05:18:54.137593 4075 ts_tablet_manager.cc:1428] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260812 05:18:54.209625 3909 tablet.cc:2369] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:18:54.229728 4080 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:18:54.229867 4080 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.230219 4080 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605)
I20260812 05:18:54.233934 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.230435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45623 (local address 127.18.166.194:57083)
0812 05:18:54.230503 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:54.230516 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:54.230571 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:54.230739 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:54.230740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:54.230742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:54.230822 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.230824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.231756 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.231757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.233004 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.233007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.233639 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.233641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.233729 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.233734 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:18:54.233790 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:18:54.233833 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":6}
I20260812 05:18:54.233955 4070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.230523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:57083 (local address 127.18.166.195:45623)
0812 05:18:54.230573 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.230576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.230588 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:18:54.230627 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.230629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.230639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.230708 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:54.230901 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.231666 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.233137 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.233555 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.233618 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.233625 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.233817 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.233864 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:18:54.234103 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.230557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:48649 (local address 127.18.166.193:35605)
0812 05:18:54.230605 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.230606 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.230787 (+ 181us) server_negotiation.cc:407] Connection header received
0812 05:18:54.230825 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.230826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.230850 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.230939 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:18:54.231089 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.231811 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.233159 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.233673 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.233749 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.233843 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.233990 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.234042 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:18:54.234326 4081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.230443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35605 (local address 127.18.166.194:48649)
0812 05:18:54.230678 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:54.230704 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 05:18:54.230760 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:18:54.230956 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:18:54.230956 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:18:54.230957 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:18:54.231011 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.231012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.231882 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.231882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.233005 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:18:54.233007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.233792 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.233793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:18:54.233867 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.233871 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:18:54.233981 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 05:18:54.234237 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":63,"thread_start_us":121,"threads_started":1}
I20260812 05:18:54.234408 4008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf01aad5b0416697afd283fddad681" candidate_uuid: "43a62db6817d40d4a113293a80d4e88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b596629beb4040458192fb57913b0620" is_pre_election: true
I20260812 05:18:54.234592 4008 raft_consensus.cc:2454] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43a62db6817d40d4a113293a80d4e88e in term 0.
I20260812 05:18:54.234643 3710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf01aad5b0416697afd283fddad681" candidate_uuid: "43a62db6817d40d4a113293a80d4e88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c091d5e70a4871bfb203384f5c2d62" is_pre_election: true
I20260812 05:18:54.234812 3710 raft_consensus.cc:2454] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43a62db6817d40d4a113293a80d4e88e in term 0.
I20260812 05:18:54.234848 3789 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a62db6817d40d4a113293a80d4e88e, b596629beb4040458192fb57913b0620; no voters:
I20260812 05:18:54.235020 4080 raft_consensus.cc:2781] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:18:54.235077 4080 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:18:54.235095 4080 raft_consensus.cc:3037] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:54.235955 4080 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.236121 4080 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [CANDIDATE]: Term 1 election: Requested vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605)
I20260812 05:18:54.236265 3710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf01aad5b0416697afd283fddad681" candidate_uuid: "43a62db6817d40d4a113293a80d4e88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c091d5e70a4871bfb203384f5c2d62"
I20260812 05:18:54.236285 4008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bf01aad5b0416697afd283fddad681" candidate_uuid: "43a62db6817d40d4a113293a80d4e88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b596629beb4040458192fb57913b0620"
I20260812 05:18:54.236372 3710 raft_consensus.cc:3037] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:54.236370 4008 raft_consensus.cc:3037] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:18:54.237135 4008 raft_consensus.cc:2454] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43a62db6817d40d4a113293a80d4e88e in term 1.
I20260812 05:18:54.237139 3710 raft_consensus.cc:2454] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43a62db6817d40d4a113293a80d4e88e in term 1.
I20260812 05:18:54.237330 3790 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a62db6817d40d4a113293a80d4e88e, 86c091d5e70a4871bfb203384f5c2d62; no voters:
I20260812 05:18:54.237442 4080 raft_consensus.cc:2781] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:18:54.237622 4080 raft_consensus.cc:686] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 1 LEADER]: Becoming Leader. State: Replica: 43a62db6817d40d4a113293a80d4e88e, State: Running, Role: LEADER
I20260812 05:18:54.237718 4080 consensus_queue.cc:237] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:18:54.238580 3551 catalog_manager.cc:5495] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e reported cstate change: term changed from 0 to 1, leader changed from <none> to 43a62db6817d40d4a113293a80d4e88e (127.18.166.194). New cstate: current_term: 1 leader_uuid: "43a62db6817d40d4a113293a80d4e88e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } health_report { overall_health: UNKNOWN } } }
I20260812 05:18:54.271029 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.268803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.18.166.194:41385)
0812 05:18:54.268846 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:18:54.268850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:18:54.268880 (+ 30us) server_negotiation.cc:407] Connection header received
0812 05:18:54.268919 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:18:54.268920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:18:54.268940 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:18:54.268998 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:18:54.269107 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.269640 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.270354 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:18:54.270540 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:18:54.270592 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:18:54.270709 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:18:54.270834 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:18:54.270897 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:18:54.270925 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 05:18:54.270944 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:18:54.272996 4008 raft_consensus.cc:1261] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Refusing update from remote peer 43a62db6817d40d4a113293a80d4e88e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:54.272997 3710 raft_consensus.cc:1261] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Refusing update from remote peer 43a62db6817d40d4a113293a80d4e88e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:18:54.273381 4082 consensus_queue.cc:1035] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:54.273564 4080 consensus_queue.cc:1035] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:18:54.278014 3549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:54.278127 3549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:18:54.278893 3549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Sending DeleteTablet for 3 replicas of tablet 16bf01aad5b0416697afd283fddad681
I20260812 05:18:54.279294 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3593
I20260812 05:18:54.279274 3987 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf01aad5b0416697afd283fddad681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:54 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:34588
I20260812 05:18:54.279323 3837 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf01aad5b0416697afd283fddad681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:54 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:42678
I20260812 05:18:54.279479 3688 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bf01aad5b0416697afd283fddad681 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:18:54 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:35176
I20260812 05:18:54.280977 4094 tablet_replica.cc:330] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: stopping tablet replica
I20260812 05:18:54.281091 4094 raft_consensus.cc:2229] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:18:54.281196 4094 raft_consensus.cc:2258] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:54.281705 4094 ts_tablet_manager.cc:1905] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:54.282621 4095 tablet_replica.cc:330] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: stopping tablet replica
I20260812 05:18:54.282766 4095 raft_consensus.cc:2229] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:18:54.282925 4095 raft_consensus.cc:2258] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:18:54.283427 4095 ts_tablet_manager.cc:1905] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:18:54.283802 4094 ts_tablet_manager.cc:1918] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:54.283885 4094 log.cc:1196] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/16bf01aad5b0416697afd283fddad681
I20260812 05:18:54.284235 4094 ts_tablet_manager.cc:1939] T 16bf01aad5b0416697afd283fddad681 P b596629beb4040458192fb57913b0620: Deleting consensus metadata
I20260812 05:18:54.284713 3536 catalog_manager.cc:4841] TS b596629beb4040458192fb57913b0620 (127.18.166.195:45623): tablet 16bf01aad5b0416697afd283fddad681 (table test-table [id=f94f60b1ed3b48f0bdf10af029fc0cdc]) successfully deleted
I20260812 05:18:54.285817 4095 ts_tablet_manager.cc:1918] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:18:54.285957 4095 log.cc:1196] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/16bf01aad5b0416697afd283fddad681
I20260812 05:18:54.286304 4095 ts_tablet_manager.cc:1939] T 16bf01aad5b0416697afd283fddad681 P 43a62db6817d40d4a113293a80d4e88e: Deleting consensus metadata
I20260812 05:18:54.286774 3536 catalog_manager.cc:4841] TS 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): tablet 16bf01aad5b0416697afd283fddad681 (table test-table [id=f94f60b1ed3b48f0bdf10af029fc0cdc]) successfully deleted
W20260812 05:18:54.287492 3537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:35605 (error 108)
I20260812 05:18:54.287967 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3765
I20260812 05:18:54.288026 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:18:54.287889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35605 (local address 127.0.0.1:35182)
0812 05:18:54.287943 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:18:54.287988 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 05:18:54.290838 3537 catalog_manager.cc:4584] TS 86c091d5e70a4871bfb203384f5c2d62 (127.18.166.193:35605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16bf01aad5b0416697afd283fddad681: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35605: connect: Connection refused (error 111)
I20260812 05:18:54.295414 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3911
I20260812 05:18:54.302070 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 3517
I20260812 05:19:14.308936 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41555
--webserver_interface=127.18.166.254
--webserver_port=40267
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:14.390722 4097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:14.391109 4097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:14.391155 4097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:14.392279 4097 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 05:19:14.392314 4097 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 05:19:14.392328 4097 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 05:19:14.392339 4097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:14.392352 4097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:14.392365 4097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:14.392377 4097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:14.392392 4097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:14.393914 4097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=40267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:14.394125 4097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:14.394309 4097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:14.399217 4106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:14.399258 4104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:14.399209 4102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:14.399715 4097 server_base.cc:1034] running on GCE node
I20260812 05:19:14.400000 4097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:14.400345 4097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:14.401471 4097 hybrid_clock.cc:648] HybridClock initialized: now 1786511954401454 us; error 47 us; skew 500 ppm
Aug 12 05:19:14 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511954, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:14.404000 4097 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:14.404495 4097 webserver.cc:460] Webserver started at http://127.18.166.254:40267/ using document root <none> and password file <none>
I20260812 05:19:14.404698 4097 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:14.404742 4097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:14.406128 4097 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:14.406787 4113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.406972 4097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:14.407061 4097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "6c4dd9bb5b724c08a88ddd155da047b3"
format_stamp: "Formatted at 2026-08-12 05:18:32 on dist-test-slave-srgv"
server_key: "9a32db58cbbacf6a5a75478c961dc2d2"
server_key_iv: "dc3fc2191825afb35c194230fac146c8"
server_key_version: "encryptionkey@0"
I20260812 05:19:14.407401 4097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:14.419395 4097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:14.421191 4097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:14.421305 4097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:14.425218 4097 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:41555
I20260812 05:19:14.425279 4168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:41555 every 8 connection(s)
I20260812 05:19:14.425632 4097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:14.427057 4169 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:19:14.427721 4169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap starting.
I20260812 05:19:14.434489 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4097
I20260812 05:19:14.434860 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:35605
--local_ip_for_outbound_sockets=127.18.166.193
--tserver_master_addrs=127.18.166.254:41555
--webserver_port=40779
--webserver_interface=127.18.166.193
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:19:14.437126 4169 log.cc:824] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:14.439453 4169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:19:14.439769 4169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Bootstrap complete.
I20260812 05:19:14.442157 4169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:19:14.442386 4169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4dd9bb5b724c08a88ddd155da047b3, State: Initialized, Role: FOLLOWER
I20260812 05:19:14.442559 4169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:19:14.442641 4169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:14.442694 4169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:14.442765 4169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 2 FOLLOWER]: Advancing to term 3
I20260812 05:19:14.443796 4169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:19:14.443918 4169 leader_election.cc:302] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c4dd9bb5b724c08a88ddd155da047b3; no voters:
I20260812 05:19:14.444151 4169 leader_election.cc:288] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 05:19:14.444187 4177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 05:19:14.444420 4177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [term 3 LEADER]: Becoming Leader. State: Replica: 6c4dd9bb5b724c08a88ddd155da047b3, State: Running, Role: LEADER
I20260812 05:19:14.444450 4169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:14.444651 4177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } }
I20260812 05:19:14.445362 4179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c4dd9bb5b724c08a88ddd155da047b3. Latest consensus state: current_term: 3 leader_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } } }
I20260812 05:19:14.445464 4179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:14.445665 4187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:14.445802 4178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dd9bb5b724c08a88ddd155da047b3" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41555 } } }
I20260812 05:19:14.445864 4178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:14.446614 4187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:14.446725 4187 catalog_manager.cc:1259] Loaded cluster ID: 7f0eed27872c42e7a30c485f0ab376f3
I20260812 05:19:14.446761 4187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:14.447866 4187 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:14.448042 4187 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Loaded TSK: 0
I20260812 05:19:14.448091 4187 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c4dd9bb5b724c08a88ddd155da047b3: Loaded TSK: 1
F20260812 05:19:14.448323 4187 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f95554199ad google::LogMessage::Fail() at ??:0
@ 0x7f955541c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f95554194de google::LogMessage::Flush() at ??:0
@ 0x7f955541aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9557b86104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9557b83d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f955d3731cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f955d37348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f955d373594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f955d339c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f955d343ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f95572e70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f95572dc954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f95581406db start_thread at ??:0
@ 0x7f9553f1761f clone at ??:0
W20260812 05:19:14.536815 4174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:14.537009 4174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:14.537031 4174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:14.538264 4174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:14.538306 4174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:14.538332 4174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:14.539835 4174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:35605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=40779
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:14.540079 4174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:14.540305 4174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:14.545257 4219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:14.545258 4218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:14.545437 4174 server_base.cc:1034] running on GCE node
W20260812 05:19:14.545258 4221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:14.545759 4174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:14.546041 4174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:14.547197 4174 hybrid_clock.cc:648] HybridClock initialized: now 1786511954547177 us; error 40 us; skew 500 ppm
Aug 12 05:19:14 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511954, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:14.549914 4174 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:19:14.550423 4174 webserver.cc:460] Webserver started at http://127.18.166.193:40779/ using document root <none> and password file <none>
I20260812 05:19:14.550673 4174 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:14.550743 4174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:14.552132 4174 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:14.552870 4230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.553468 4174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:14.553568 4174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "86c091d5e70a4871bfb203384f5c2d62"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "b060d8599f52e6e92b1cd1b3b8284309"
server_key_iv: "4ea18c736425715b4195a8219041c0f6"
server_key_version: "encryptionkey@0"
I20260812 05:19:14.553930 4174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:14.563005 4174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:14.565307 4174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:14.565500 4174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:14.565740 4174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:14.566181 4237 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:19:14.568063 4174 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:19:14.568122 4174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 05:19:14.568184 4174 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:19:14.568800 4174 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:19:14.568857 4174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:19:14.568926 4237 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap starting.
I20260812 05:19:14.576231 4174 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:35605
I20260812 05:19:14.576385 4352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:35605 every 8 connection(s)
I20260812 05:19:14.576650 4174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:19:14.580252 4237 log.cc:824] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:14.580749 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4174
I20260812 05:19:14.581017 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:41385
--local_ip_for_outbound_sockets=127.18.166.194
--tserver_master_addrs=127.18.166.254:41555
--webserver_port=33431
--webserver_interface=127.18.166.194
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:19:14.583081 4237 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:19:14.583416 4237 tablet_bootstrap.cc:492] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Bootstrap complete.
I20260812 05:19:14.584915 4237 ts_tablet_manager.cc:1397] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260812 05:19:14.586494 4237 raft_consensus.cc:348] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:14.586833 4237 raft_consensus.cc:729] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c091d5e70a4871bfb203384f5c2d62, State: Initialized, Role: FOLLOWER
I20260812 05:19:14.587016 4237 consensus_queue.cc:260] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:14.587316 4237 ts_tablet_manager.cc:1428] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 05:19:14.646565 4355 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:19:14.577347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:41689)
0812 05:19:14.577579 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.577648 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:14.578393 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:14.646343 (+ 67950us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":143,"threads_started":1}
W20260812 05:19:14.646867 4353 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:41555 (0 consecutive failures): Network error: Failed to ping master at 127.18.166.254:41555: Client connection negotiation failed: client connection to 127.18.166.254:41555: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:19:14.647264 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.647145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:51315)
0812 05:19:14.647202 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.647243 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:19:14.647643 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.647546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:33515)
0812 05:19:14.647597 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.647623 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 05:19:14.647776 4353 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:19:14.666270 4357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:14.666507 4357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:14.666546 4357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:14.667691 4357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:14.667762 4357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:14.667809 4357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:14.669279 4357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:41385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=33431
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:14.669567 4357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:14.669804 4357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:14.674800 4363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:14.674860 4369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:14.674840 4357 server_base.cc:1034] running on GCE node
W20260812 05:19:14.674798 4364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:14.675256 4357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:14.675505 4357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:14.676673 4357 hybrid_clock.cc:648] HybridClock initialized: now 1786511954676665 us; error 27 us; skew 500 ppm
Aug 12 05:19:14 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511954, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:14.679234 4357 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:19:14.679612 4357 webserver.cc:460] Webserver started at http://127.18.166.194:33431/ using document root <none> and password file <none>
I20260812 05:19:14.679821 4357 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:14.679862 4357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:14.681115 4357 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:14.681772 4376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.681944 4357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.682047 4357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "43a62db6817d40d4a113293a80d4e88e"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "ffc77d3824074b7ad06ec4d74182fa71"
server_key_iv: "e8dcee92aa05f7ea488e80486c3fdf0c"
server_key_version: "encryptionkey@0"
I20260812 05:19:14.682394 4357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:14.700362 4357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:14.702368 4357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:14.702536 4357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:14.702780 4357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:14.703090 4357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:14.703146 4357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.703182 4357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:14.703231 4357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.709239 4357 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:41385
I20260812 05:19:14.709309 4492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:41385 every 8 connection(s)
I20260812 05:19:14.709705 4357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:19:14.710731 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.710360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:34403)
0812 05:19:14.710521 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.710590 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
W20260812 05:19:14.711174 4493 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:41555 (0 consecutive failures): Network error: Failed to ping master at 127.18.166.254:41555: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
I20260812 05:19:14.711436 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.711347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:59165)
0812 05:19:14.711392 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.711418 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 05:19:14.711731 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.711654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:35133)
0812 05:19:14.711694 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.711714 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 05:19:14.711848 4493 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:19:14.718469 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4357
I20260812 05:19:14.718757 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:45623
--local_ip_for_outbound_sockets=127.18.166.195
--tserver_master_addrs=127.18.166.254:41555
--webserver_port=34791
--webserver_interface=127.18.166.195
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:14.799330 4496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:14.799562 4496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:14.799603 4496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:14.800746 4496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:14.800812 4496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:14.800858 4496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:14.802347 4496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:45623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=34791
--tserver_master_addrs=127.18.166.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:14.802614 4496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:14.802852 4496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:14.807811 4504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:14.807811 4502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:14.807910 4501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:14.807927 4496 server_base.cc:1034] running on GCE node
I20260812 05:19:14.808261 4496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:14.808503 4496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:14.809680 4496 hybrid_clock.cc:648] HybridClock initialized: now 1786511954809658 us; error 38 us; skew 500 ppm
Aug 12 05:19:14 dist-test-slave-srgv krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511954, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:14.812183 4496 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:19:14.812641 4496 webserver.cc:460] Webserver started at http://127.18.166.195:34791/ using document root <none> and password file <none>
I20260812 05:19:14.812879 4496 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:14.812950 4496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:14.814321 4496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:14.814922 4513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.815094 4496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:14.815193 4496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "b596629beb4040458192fb57913b0620"
format_stamp: "Formatted at 2026-08-12 05:18:33 on dist-test-slave-srgv"
server_key: "58b8dc3db1da2e11610ab008aaf8b0d6"
server_key_iv: "cdbdca5f168a38730301997f3f4d21e7"
server_key_version: "encryptionkey@0"
I20260812 05:19:14.815544 4496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:14.830574 4496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:14.833276 4496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:14.833501 4496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:14.833760 4496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:14.834079 4496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:14.834142 4496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.834187 4496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:14.834213 4496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:14.840160 4496 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:45623
I20260812 05:19:14.840212 4632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:45623 every 8 connection(s)
I20260812 05:19:14.840551 4496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:19:14.841630 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.841212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:33215)
0812 05:19:14.841366 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.841481 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
W20260812 05:19:14.842072 4633 heartbeater.cc:644] Failed to heartbeat to 127.18.166.254:41555 (0 consecutive failures): Network error: Failed to ping master at 127.18.166.254:41555: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
I20260812 05:19:14.842360 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.842259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:58487)
0812 05:19:14.842313 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.842342 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:19:14.842612 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:14.842561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:41741)
0812 05:19:14.842583 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:14.842600 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 05:19:14.842723 4633 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:19:14.843753 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4496
I20260812 05:19:15.648407 4637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.648134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:54129)
0812 05:19:15.648298 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:15.648339 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260812 05:19:15.712349 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.712135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:54165)
0812 05:19:15.712265 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:15.712299 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260812 05:19:15.843243 4639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.843053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:41573)
0812 05:19:15.843165 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:15.843201 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260812 05:19:15.970891 4640 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:15.971019 4640 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:15.971415 4640 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
Aug 12 05:19:15 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511954, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.195@KRBTEST.COM
Aug 12 05:19:15 dist-test-slave-srgv krb5kdc[2906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511954, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.194@KRBTEST.COM
I20260812 05:19:15.983780 4637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.971566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:45623 (local address 127.18.166.193:40773)
0812 05:19:15.971628 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:15.971640 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:15.971710 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:15.972874 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:15.972877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:15.973425 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:15.973704 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:15.973712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.975357 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.975357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:15.976682 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:15.976685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.977511 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.977512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:15.977610 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:15.978286 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:15.978297 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:15.979932 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:15.982222 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:15.982230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:15.982441 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:15.982720 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:15.982721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:15.982837 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:15.983248 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:15.983255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:15.983380 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 05:19:15.983481 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 05:19:15.983681 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":107}
I20260812 05:19:15.983799 4642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.971687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:40773 (local address 127.18.166.195:45623)
0812 05:19:15.971931 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:15.971944 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:15.971982 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:19:15.972587 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:15.972591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:15.972650 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:15.972808 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:15.973835 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.975240 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.976776 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.977427 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.977540 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:15.980042 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:15.980046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:15.980046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:15.980056 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:15.982098 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982536 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:15.982539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:15.982540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:15.982638 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982919 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:15.982920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:15.982920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:15.983161 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:15.983267 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:15.983544 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 05:19:15.983670 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260812 05:19:15.983916 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.971691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:41385 (local address 127.18.166.193:33757)
0812 05:19:15.971807 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:15.971820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:15.971884 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:15.972874 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:15.972877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:15.973429 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:15.973704 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:15.973712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.975356 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.975357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:15.976682 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:15.976685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.977652 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.977654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:15.977710 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:15.978328 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:15.978332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:15.980330 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:15.982469 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:15.982471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:15.982644 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:15.982891 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:15.982892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:15.982955 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:15.983335 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:15.983336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:15.983382 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 05:19:15.983479 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:19:15.983761 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":203,"thread_start_us":53,"threads_started":1}
I20260812 05:19:15.983801 4643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:15.971820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:33757 (local address 127.18.166.194:41385)
0812 05:19:15.971970 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:15.971977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:15.972005 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:19:15.972587 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:15.972591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:15.972650 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:15.972808 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:15.973835 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.975244 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.976775 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:15.977425 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:15.977540 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:15.980462 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:15.980465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:15.980466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:15.980475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:15.982387 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:15.982726 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:15.982729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:15.982730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:15.982818 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:15.983021 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:15.983022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:15.983023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:15.983258 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:15.983340 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:15.983532 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 05:19:15.983670 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":54,"threads_started":1}
W20260812 05:19:15.984763 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b596629beb4040458192fb57913b0620 (127.18.166.195:45623): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
W20260812 05:19:15.985203 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
I20260812 05:19:15.985261 4232 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c091d5e70a4871bfb203384f5c2d62; no voters: 43a62db6817d40d4a113293a80d4e88e, b596629beb4040458192fb57913b0620
I20260812 05:19:15.985440 4640 raft_consensus.cc:2726] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:19:16.649154 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:16.648901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:37673)
0812 05:19:16.649029 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:16.649100 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260812 05:19:16.712996 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:16.712793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:45965)
0812 05:19:16.712919 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:16.712957 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260812 05:19:16.843963 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:16.843713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:33495)
0812 05:19:16.843873 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:16.843924 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":60,"threads_started":1}
I20260812 05:19:17.649899 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:17.649683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:44527)
0812 05:19:17.649812 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:17.649853 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260812 05:19:17.713784 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:17.713560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:57033)
0812 05:19:17.713696 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:17.713741 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260812 05:19:17.740154 4650 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:17.740270 4650 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:17.740499 4650 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
W20260812 05:19:17.741015 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
W20260812 05:19:17.741097 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b596629beb4040458192fb57913b0620 (127.18.166.195:45623): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
I20260812 05:19:17.741132 4232 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c091d5e70a4871bfb203384f5c2d62; no voters: 43a62db6817d40d4a113293a80d4e88e, b596629beb4040458192fb57913b0620
I20260812 05:19:17.741242 4650 raft_consensus.cc:2726] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:19:17.845297 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:17.845079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:41503)
0812 05:19:17.845217 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:17.845257 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260812 05:19:18.650549 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:18.650348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:53371)
0812 05:19:18.650465 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:18.650502 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260812 05:19:18.714473 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:18.714243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:35875)
0812 05:19:18.714383 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:18.714427 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260812 05:19:18.846123 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:18.845867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:33129)
0812 05:19:18.846031 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:18.846078 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":65,"threads_started":1}
I20260812 05:19:19.651407 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:19.651163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:33183)
0812 05:19:19.651312 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:19.651359 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260812 05:19:19.715240 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:19.714995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:56481)
0812 05:19:19.715137 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:19.715177 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260812 05:19:19.847021 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:19.846782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:58151)
0812 05:19:19.846932 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:19.846975 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260812 05:19:20.652324 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:20.652079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:53483)
0812 05:19:20.652236 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:20.652279 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260812 05:19:20.716104 4659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:20.715822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:33945)
0812 05:19:20.715997 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:20.716047 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260812 05:19:20.847716 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:20.847506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:36337)
0812 05:19:20.847631 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:20.847665 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260812 05:19:20.893623 4661 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:20.893734 4661 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:20.893887 4661 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
W20260812 05:19:20.894363 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b596629beb4040458192fb57913b0620 (127.18.166.195:45623): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
W20260812 05:19:20.894466 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
I20260812 05:19:20.894484 4232 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c091d5e70a4871bfb203384f5c2d62; no voters: 43a62db6817d40d4a113293a80d4e88e, b596629beb4040458192fb57913b0620
I20260812 05:19:20.894624 4661 raft_consensus.cc:2726] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:19:21.653167 4662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:21.652821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:58403)
0812 05:19:21.653070 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:21.653118 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":182,"threads_started":1}
I20260812 05:19:21.716810 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:21.716603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:58169)
0812 05:19:21.716732 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:21.716768 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260812 05:19:21.848320 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:21.848116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:57215)
0812 05:19:21.848238 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:21.848276 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260812 05:19:22.653946 4665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:22.653713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:32947)
0812 05:19:22.653854 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:22.653900 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260812 05:19:22.717605 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:22.717304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:37363)
0812 05:19:22.717516 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:22.717557 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260812 05:19:22.848980 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:22.848751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:53461)
0812 05:19:22.848896 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:22.848935 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260812 05:19:23.654706 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:23.654450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:52105)
0812 05:19:23.654612 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:23.654659 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260812 05:19:23.718290 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:23.718073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:35593)
0812 05:19:23.718190 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:23.718234 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
I20260812 05:19:23.849761 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:23.849558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:38579)
0812 05:19:23.849681 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:23.849718 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260812 05:19:24.655534 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:24.655278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:36871)
0812 05:19:24.655438 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:24.655488 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260812 05:19:24.718991 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:24.718778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:59329)
0812 05:19:24.718904 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:24.718947 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260812 05:19:24.850493 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:24.850246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:55295)
0812 05:19:24.850401 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:24.850445 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":66,"threads_started":1}
I20260812 05:19:25.322928 4675 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:25.323029 4675 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:25.323195 4675 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
W20260812 05:19:25.323617 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b596629beb4040458192fb57913b0620 (127.18.166.195:45623): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
W20260812 05:19:25.323678 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
I20260812 05:19:25.323693 4232 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c091d5e70a4871bfb203384f5c2d62; no voters: 43a62db6817d40d4a113293a80d4e88e, b596629beb4040458192fb57913b0620
I20260812 05:19:25.323792 4675 raft_consensus.cc:2726] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:19:25.656369 4676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:25.656151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:45645)
0812 05:19:25.656277 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:25.656321 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260812 05:19:25.719622 4678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:25.719394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:46263)
0812 05:19:25.719532 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:25.719566 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260812 05:19:25.851265 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:25.851032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:51045)
0812 05:19:25.851182 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:25.851219 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260812 05:19:26.657013 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:26.656801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:37469)
0812 05:19:26.656912 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:26.656951 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260812 05:19:26.720306 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:26.720074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:59527)
0812 05:19:26.720209 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:26.720253 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260812 05:19:26.852154 4682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:26.851902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:44091)
0812 05:19:26.852052 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:26.852101 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260812 05:19:27.657717 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:27.657495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:36807)
0812 05:19:27.657625 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:27.657663 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260812 05:19:27.720969 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:27.720755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:51293)
0812 05:19:27.720882 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:27.720922 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260812 05:19:27.852895 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:27.852631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:60101)
0812 05:19:27.852801 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:27.852846 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260812 05:19:28.588559 4686 raft_consensus.cc:482] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:28.588656 4686 raft_consensus.cc:504] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b596629beb4040458192fb57913b0620" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 45623 } } peers { permanent_uuid: "43a62db6817d40d4a113293a80d4e88e" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 41385 } } peers { permanent_uuid: "86c091d5e70a4871bfb203384f5c2d62" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35605 } }
I20260812 05:19:28.588864 4686 leader_election.cc:288] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b596629beb4040458192fb57913b0620 (127.18.166.195:45623), 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385)
W20260812 05:19:28.589304 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b596629beb4040458192fb57913b0620 (127.18.166.195:45623): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
W20260812 05:19:28.589385 4232 leader_election.cc:341] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43a62db6817d40d4a113293a80d4e88e (127.18.166.194:41385): Not found: Tablet not found: 16bf01aad5b0416697afd283fddad681
I20260812 05:19:28.589437 4232 leader_election.cc:302] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86c091d5e70a4871bfb203384f5c2d62; no voters: 43a62db6817d40d4a113293a80d4e88e, b596629beb4040458192fb57913b0620
I20260812 05:19:28.589512 4686 raft_consensus.cc:2726] T 16bf01aad5b0416697afd283fddad681 P 86c091d5e70a4871bfb203384f5c2d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:19:28.658335 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:28.658128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:34715)
0812 05:19:28.658254 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:28.658290 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260812 05:19:28.721696 4688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:28.721399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:41181)
0812 05:19:28.721563 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:28.721613 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260812 05:19:28.853704 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:28.853470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:34459)
0812 05:19:28.853619 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:28.853659 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260812 05:19:29.659157 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:29.658898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.193:56531)
0812 05:19:29.659066 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:29.659112 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260812 05:19:29.722355 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:29.722131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.194:47389)
0812 05:19:29.722266 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:29.722308 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260812 05:19:29.845348 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 4174
I20260812 05:19:29.851629 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 4357
I20260812 05:19:29.854426 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:29.854227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:41555 (local address 127.18.166.195:40367)
0812 05:19:29.854351 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:29.854385 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.254:41555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260812 05:19:29.857173 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 4496
2026-08-12T05:19:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58344 ms)
[----------] 30 tests from SecurityITest (238692 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:29 dist-test-slave-srgv krb5kdc[4695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:29 dist-test-slave-srgv krb5kdc[4695](info): set up 2 sockets
Aug 12 05:19:29 dist-test-slave-srgv krb5kdc[4695](info): commencing operation
krb5kdc: starting...
W20260812 05:19:30.934458 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:19:31.058393 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42795
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:42795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:31.140465 4708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:31.140673 4708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:31.140697 4708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:31.141876 4708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:31.141919 4708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:31.141934 4708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:31.141947 4708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:31.141959 4708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:31.143532 4708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:42795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:31.143793 4708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:31.144023 4708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:31.149238 4713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:31.149310 4708 server_base.cc:1034] running on GCE node
W20260812 05:19:31.149267 4716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:31.149278 4714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:31.149756 4708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:31.150055 4708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:31.151221 4708 hybrid_clock.cc:648] HybridClock initialized: now 1786511971151206 us; error 28 us; skew 500 ppm
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:31.153716 4708 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:31.154178 4708 webserver.cc:460] Webserver started at http://127.18.166.254:36279/ using document root <none> and password file <none>
I20260812 05:19:31.154408 4708 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:31.154489 4708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:31.154651 4708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:31.155702 4708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "cb90267a27e74b408ed3287073fb63bf"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "923724542185331ad8cfa0d6c4a5f63e"
server_key_iv: "dbb0108f0ed2f0d41694fb43e1e40325"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.156067 4708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "cb90267a27e74b408ed3287073fb63bf"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "923724542185331ad8cfa0d6c4a5f63e"
server_key_iv: "dbb0108f0ed2f0d41694fb43e1e40325"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.157400 4708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:31.158192 4724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.158361 4708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:31.158458 4708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "cb90267a27e74b408ed3287073fb63bf"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "923724542185331ad8cfa0d6c4a5f63e"
server_key_iv: "dbb0108f0ed2f0d41694fb43e1e40325"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.158540 4708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:31.181958 4708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:31.184005 4708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:31.184185 4708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:31.188335 4708 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:42795
I20260812 05:19:31.188375 4781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:42795 every 8 connection(s)
I20260812 05:19:31.188766 4708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:31.189796 4782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:31.192168 4782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: Bootstrap starting.
I20260812 05:19:31.192751 4782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:31.193006 4782 log.cc:824] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:31.193773 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4708
I20260812 05:19:31.193883 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:31.193939 4782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: No bootstrap required, opened a new log
I20260812 05:19:31.194175 19099 external_mini_cluster.cc:1428] Setting key b81d0e7e0baf1930f2e58afcee8fdc14
I20260812 05:19:31.196604 4782 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb90267a27e74b408ed3287073fb63bf" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42795 } }
I20260812 05:19:31.196751 4782 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:31.196784 4782 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb90267a27e74b408ed3287073fb63bf, State: Initialized, Role: FOLLOWER
I20260812 05:19:31.196923 4782 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb90267a27e74b408ed3287073fb63bf" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42795 } }
I20260812 05:19:31.197002 4782 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:31.197027 4782 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:31.197093 4782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:31.197796 4782 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb90267a27e74b408ed3287073fb63bf" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42795 } }
I20260812 05:19:31.197914 4782 leader_election.cc:302] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb90267a27e74b408ed3287073fb63bf; no voters:
I20260812 05:19:31.198113 4782 leader_election.cc:288] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:31.198199 4787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:31.198390 4782 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:31.198403 4787 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [term 1 LEADER]: Becoming Leader. State: Replica: cb90267a27e74b408ed3287073fb63bf, State: Running, Role: LEADER
I20260812 05:19:31.198702 4787 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb90267a27e74b408ed3287073fb63bf" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42795 } }
I20260812 05:19:31.200376 4788 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb90267a27e74b408ed3287073fb63bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb90267a27e74b408ed3287073fb63bf" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42795 } } }
I20260812 05:19:31.201542 4788 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:31.201539 4789 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb90267a27e74b408ed3287073fb63bf. Latest consensus state: current_term: 1 leader_uuid: "cb90267a27e74b408ed3287073fb63bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb90267a27e74b408ed3287073fb63bf" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42795 } } }
I20260812 05:19:31.201696 4789 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf [sys.catalog]: This master's current role is: LEADER
W20260812 05:19:31.200449 4802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:19:31.201880 4802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:19:31.202023 4804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:31.202240 4804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:31.204102 4804 catalog_manager.cc:1347] Generated new cluster ID: ab0a00eb1d144fd48bbe7eaa9bacf57e
I20260812 05:19:31.204162 4804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:31.207453 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.194769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.18.166.254:42795)
0812 05:19:31.194930 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.194944 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.194971 (+ 27us) server_negotiation.cc:407] Connection header received
0812 05:19:31.195440 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.195448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.195498 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.195631 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:31.196462 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.197817 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.198921 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.199498 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.199638 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.202739 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:31.202743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:31.202744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:31.202768 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:31.205052 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.205524 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.205528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.205529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.205669 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.205883 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.205884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.205884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.206229 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:31.206964 (+ 735us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.207050 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.207223 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260812 05:19:31.223083 4804 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:31.223870 4804 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:31.230109 4804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: Generated new TSK 0
I20260812 05:19:31.230365 4804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:19:31.258333 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42795
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:31.341917 4808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:31.342131 4808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:31.342156 4808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:31.343341 4808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:31.343389 4808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:31.343415 4808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:31.344810 4808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:31.344992 4808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:31.345191 4808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:31.350309 4814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:31.350313 4813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:31.350488 4808 server_base.cc:1034] running on GCE node
W20260812 05:19:31.350309 4816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:31.350776 4808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:31.351027 4808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:31.352169 4808 hybrid_clock.cc:648] HybridClock initialized: now 1786511971352157 us; error 31 us; skew 500 ppm
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:31.354815 4808 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:19:31.355252 4808 webserver.cc:460] Webserver started at http://127.18.166.193:40965/ using document root <none> and password file <none>
I20260812 05:19:31.355473 4808 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:31.355520 4808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:31.355693 4808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:31.356753 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "28a881ccf1a74183a3f51934a7a3afa1"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "b68ae71a8467865bf26336720cf302b8"
server_key_iv: "f37da4bca9c2d3be21771dbc25e2eec5"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.357115 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "28a881ccf1a74183a3f51934a7a3afa1"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "b68ae71a8467865bf26336720cf302b8"
server_key_iv: "f37da4bca9c2d3be21771dbc25e2eec5"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.358510 4808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:19:31.359261 4824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.359433 4808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:31.359524 4808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "28a881ccf1a74183a3f51934a7a3afa1"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "b68ae71a8467865bf26336720cf302b8"
server_key_iv: "f37da4bca9c2d3be21771dbc25e2eec5"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.359596 4808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:31.377116 4808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:31.379686 4808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:31.379848 4808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:31.380091 4808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:31.380415 4808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:31.380466 4808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.380509 4808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:31.380546 4808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.386566 4808 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:43809
I20260812 05:19:31.386682 4946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:43809 every 8 connection(s)
I20260812 05:19:31.386974 4808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:31.393661 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4808
I20260812 05:19:31.393800 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:19:31.394039 19099 external_mini_cluster.cc:1428] Setting key 9ca0cd30ae4dac71d8491c5826d92892
I20260812 05:19:31.397106 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.387764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:47169 (local address 127.18.166.254:42795)
0812 05:19:31.387848 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.387850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.388098 (+ 248us) server_negotiation.cc:407] Connection header received
0812 05:19:31.388562 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.388563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.388574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.388637 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:31.389598 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.390142 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.391260 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.391496 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.391565 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.393816 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:31.393818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:31.393818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:31.393825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:31.395592 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.395987 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.395989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.395989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.396057 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.396359 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.396360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.396361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.396524 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:31.396606 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.396953 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.397039 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:19:31.397466 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.387710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42795 (local address 127.18.166.193:47169)
0812 05:19:31.387920 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:31.387960 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:31.388449 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:31.388698 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:31.388702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:31.389287 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:31.389495 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.389500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.390235 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.390236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.391034 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.391036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.391583 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.391584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.391685 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.392164 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:31.392172 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:31.393705 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:31.395678 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:31.395685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:31.395687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:31.395914 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:31.396180 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:31.396180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:31.396180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:31.396285 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:31.396610 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:31.396618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:31.396796 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 05:19:31.396928 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 05:19:31.397164 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
I20260812 05:19:31.398029 4947 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42795
I20260812 05:19:31.398140 4947 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:31.398343 4947 heartbeater.cc:511] Master 127.18.166.254:42795 requested a full tablet report, sending...
I20260812 05:19:31.398874 4742 ts_manager.cc:194] Registered new tserver with Master: 28a881ccf1a74183a3f51934a7a3afa1 (127.18.166.193:43809)
I20260812 05:19:31.399741 4742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:47169
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:19:31.422361 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42795
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:31.514477 4952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:31.514787 4952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:31.514835 4952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:31.516897 4952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:31.516974 4952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:31.517025 4952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:31.518622 4952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:31.518908 4952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:31.519155 4952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:31.525663 4957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:31.525761 4958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:31.525655 4960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:31.529991 4952 server_base.cc:1034] running on GCE node
I20260812 05:19:31.530253 4952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:31.530592 4952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:31.531769 4952 hybrid_clock.cc:648] HybridClock initialized: now 1786511971531747 us; error 44 us; skew 500 ppm
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:31.534827 4952 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:19:31.535454 4952 webserver.cc:460] Webserver started at http://127.18.166.194:43571/ using document root <none> and password file <none>
I20260812 05:19:31.535790 4952 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:31.535863 4952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:31.536032 4952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:31.537691 4952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "8545f49f92a84b7c9d1dfa935207c43d"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "1d61a319cc671d7004efd2327c6b05a6"
server_key_iv: "486590bf8f6d7692b887bf5fb1d52c76"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.538160 4952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "8545f49f92a84b7c9d1dfa935207c43d"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "1d61a319cc671d7004efd2327c6b05a6"
server_key_iv: "486590bf8f6d7692b887bf5fb1d52c76"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.539623 4952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:19:31.540329 4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.540500 4952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:31.540606 4952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "8545f49f92a84b7c9d1dfa935207c43d"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "1d61a319cc671d7004efd2327c6b05a6"
server_key_iv: "486590bf8f6d7692b887bf5fb1d52c76"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.540693 4952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:31.557235 4952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:31.559367 4952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:31.559544 4952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:31.559803 4952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:31.560134 4952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:31.560187 4952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.560230 4952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:31.560262 4952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.566499 4952 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:43383
I20260812 05:19:31.566545 5089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:43383 every 8 connection(s)
I20260812 05:19:31.566875 4952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:19:31.567818 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 4952
I20260812 05:19:31.567941 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:19:31.568212 19099 external_mini_cluster.cc:1428] Setting key 374b8933e64d375a2ec5f81856412f8c
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:19:31.577541 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.567693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:50463 (local address 127.18.166.254:42795)
0812 05:19:31.567744 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.567746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.568043 (+ 297us) server_negotiation.cc:407] Connection header received
0812 05:19:31.568675 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.568676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.568689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.568752 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:31.569816 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.570598 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.571990 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.572322 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.572398 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.574682 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:31.574683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:31.574684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:31.574691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:31.576081 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.576503 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.576505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.576505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.576564 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.576831 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.576832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.576832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.576951 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:31.577030 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.577336 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.577445 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:19:31.577749 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.567684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42795 (local address 127.18.166.194:50463)
0812 05:19:31.567871 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:31.567934 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:31.568573 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:31.568847 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:31.568851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:31.569529 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:31.569726 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.569730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.570701 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.570702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.571906 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.571909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.572408 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.572409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.572509 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.573096 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:31.573105 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:31.574573 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:31.576170 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:31.576178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:31.576181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:31.576412 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:31.576645 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:31.576646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:31.576647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:31.576758 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:31.577019 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:31.577027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:31.577201 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 05:19:31.577312 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 05:19:31.577506 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260812 05:19:31.578266 5090 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42795
I20260812 05:19:31.578380 5090 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:31.578577 5090 heartbeater.cc:511] Master 127.18.166.254:42795 requested a full tablet report, sending...
I20260812 05:19:31.578969 4742 ts_manager.cc:194] Registered new tserver with Master: 8545f49f92a84b7c9d1dfa935207c43d (127.18.166.194:43383)
I20260812 05:19:31.579394 4742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:50463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:19:31.596503 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42795
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:31.682605 5095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:31.682847 5095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:31.682889 5095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:31.684623 5095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:31.684700 5095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:31.684756 5095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:31.686547 5095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:31.686831 5095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:31.687081 5095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:31.692456 5105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:31.692683 5103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:31.692538 5102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:31.692790 5095 server_base.cc:1034] running on GCE node
I20260812 05:19:31.692986 5095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:31.693246 5095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:31.694413 5095 hybrid_clock.cc:648] HybridClock initialized: now 1786511971694400 us; error 30 us; skew 500 ppm
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:31.697029 5095 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:19:31.697687 5095 webserver.cc:460] Webserver started at http://127.18.166.195:37917/ using document root <none> and password file <none>
I20260812 05:19:31.697945 5095 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:31.698019 5095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:31.698194 5095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:31.699443 5095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "fed74f0a46e649da9f44c2ede15f914e"
server_key_iv: "1cb41c33440b5301d7b5be0a7b3924e4"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.699829 5095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "fed74f0a46e649da9f44c2ede15f914e"
server_key_iv: "1cb41c33440b5301d7b5be0a7b3924e4"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.701319 5095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:31.702301 5113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.702519 5095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:31.702630 5095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01"
format_stamp: "Formatted at 2026-08-12 05:19:31 on dist-test-slave-srgv"
server_key: "fed74f0a46e649da9f44c2ede15f914e"
server_key_iv: "1cb41c33440b5301d7b5be0a7b3924e4"
server_key_version: "encryptionkey@0"
I20260812 05:19:31.702721 5095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:31.718128 5095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:31.720737 5095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:31.720913 5095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:31.721171 5095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:31.721560 5095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:31.721630 5095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.721674 5095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:31.721701 5095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:31.728798 5095 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:36697
I20260812 05:19:31.728844 5232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:36697 every 8 connection(s)
I20260812 05:19:31.729197 5095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:19:31.732055 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5095
I20260812 05:19:31.732177 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:19:31.732362 19099 external_mini_cluster.cc:1428] Setting key d4fd65206ccc63f0b56ee8c7cb75bb64
Aug 12 05:19:31 dist-test-slave-srgv krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:31.739295 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.729973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:44551 (local address 127.18.166.254:42795)
0812 05:19:31.730035 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.730038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.730305 (+ 267us) server_negotiation.cc:407] Connection header received
0812 05:19:31.730810 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.730811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.730823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.730877 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:31.731866 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.732675 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.734048 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.734244 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.734310 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.736553 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:31.736554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:31.736554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:31.736561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:31.738031 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.738435 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.738436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.738437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.738495 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.738753 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.738753 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.738754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.738864 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:31.738919 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.739159 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.739230 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:19:31.739485 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.729926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42795 (local address 127.18.166.195:44551)
0812 05:19:31.730102 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:31.730183 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:31.730719 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:31.730930 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:31.730933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:31.731525 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:31.731722 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.731727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.732805 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.732806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.733951 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.733954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.734329 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.734330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.734435 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.734950 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:31.734959 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:31.736441 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:31.738107 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:31.738114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:31.738115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:31.738350 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:31.738572 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:31.738572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:31.738573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:31.738683 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:31.738922 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:31.738926 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:31.739041 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 05:19:31.739125 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:19:31.739244 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260812 05:19:31.740048 5233 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42795
I20260812 05:19:31.740176 5233 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:31.740331 5233 heartbeater.cc:511] Master 127.18.166.254:42795 requested a full tablet report, sending...
I20260812 05:19:31.740887 4742 ts_manager.cc:194] Registered new tserver with Master: 7ed81e3b51c94bdc9dd03b5e640dfa01 (127.18.166.195:36697)
I20260812 05:19:31.741285 4742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:44551
I20260812 05:19:31.744259 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:19:31.751819 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.745773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.18.166.254:42795)
0812 05:19:31.745837 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.745840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.745992 (+ 152us) server_negotiation.cc:407] Connection header received
0812 05:19:31.746029 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.746030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.746038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.746077 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:31.746611 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.747082 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.747833 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.748021 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.748087 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.749388 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:31.749390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:31.749390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:31.749396 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:31.750731 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.751061 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.751062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.751063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.751117 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:31.751319 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:31.751319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:31.751320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:31.751470 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:31.751548 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.751651 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.751711 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:19:31.753626 4742 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:19:31.755336 4742 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:19:31.762399 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.757803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:36697 (local address 127.0.0.1:40560)
0812 05:19:31.758271 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:31.758284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:31.758343 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:31.758573 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:31.758575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:31.758578 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:31.758709 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.758713 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.760023 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.760024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.761151 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.761153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.762067 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.762068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.762155 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.762171 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:19:31.762253 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 05:19:31.762324 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":39,"thread_start_us":118,"threads_started":1}
I20260812 05:19:31.762411 5250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.758073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.18.166.195:36697)
0812 05:19:31.758382 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.758386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.758396 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:31.758437 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.758439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.758454 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.758652 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:31.758783 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.759911 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.761247 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.761982 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.762107 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.762185 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.762267 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.762313 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":101,"threads_started":1}
I20260812 05:19:31.762439 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.758073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.18.166.194:43383)
0812 05:19:31.758279 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.758284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.758524 (+ 240us) server_negotiation.cc:407] Connection header received
0812 05:19:31.758569 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.758571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.758585 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.758654 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:31.758874 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.759961 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.761253 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.761982 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.762058 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.762119 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.762311 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.762350 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260812 05:19:31.762537 5248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.757909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.18.166.193:43809)
0812 05:19:31.758309 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:31.758314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:31.758325 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:19:31.758369 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:31.758371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:31.758389 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:31.758473 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:31.758907 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.759577 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.761137 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.761983 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.762319 (+ 336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.762381 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:31.762405 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 05:19:31.762450 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":84,"threads_started":1}
I20260812 05:19:31.762734 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.757812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:43383 (local address 127.0.0.1:53166)
0812 05:19:31.758413 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:31.758423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:31.758466 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:31.758740 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:31.758741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:31.758743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:31.758790 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.758791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.760074 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.760076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.761157 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.761160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.762148 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.762149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.762209 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.762214 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:19:31.762265 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 05:19:31.762652 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":327,"thread_start_us":98,"threads_started":1}
I20260812 05:19:31.762969 5247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:31.757788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:43809 (local address 127.0.0.1:42828)
0812 05:19:31.758092 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:31.758138 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:31.758222 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:31.758572 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:31.758575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:31.758577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:31.758821 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.758823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.759703 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.759704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.761045 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:31.761048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:31.762140 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:31.762141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:31.762220 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:31.762224 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:31.762265 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 05:19:31.762871 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":596,"thread_start_us":68,"threads_started":1}
I20260812 05:19:31.763667 5019 tablet_service.cc:1467] Processing CreateTablet for tablet 94574f5800a44311b7ad18c2ab49a699 (DEFAULT_TABLE table=test-table [id=cf367db2974b4627bfc0c689fe06d28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:31.763924 5164 tablet_service.cc:1467] Processing CreateTablet for tablet 94574f5800a44311b7ad18c2ab49a699 (DEFAULT_TABLE table=test-table [id=cf367db2974b4627bfc0c689fe06d28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:31.764042 5019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94574f5800a44311b7ad18c2ab49a699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:31.764261 5164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94574f5800a44311b7ad18c2ab49a699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:31.766970 5252 tablet_bootstrap.cc:492] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Bootstrap starting.
I20260812 05:19:31.767683 5252 tablet_bootstrap.cc:654] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:31.767617 4877 tablet_service.cc:1467] Processing CreateTablet for tablet 94574f5800a44311b7ad18c2ab49a699 (DEFAULT_TABLE table=test-table [id=cf367db2974b4627bfc0c689fe06d28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:31.767990 4877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94574f5800a44311b7ad18c2ab49a699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:31.768296 5252 log.cc:824] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:31.768669 5251 tablet_bootstrap.cc:492] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Bootstrap starting.
I20260812 05:19:31.769207 5252 tablet_bootstrap.cc:492] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: No bootstrap required, opened a new log
I20260812 05:19:31.769282 5252 ts_tablet_manager.cc:1397] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:31.769322 5251 tablet_bootstrap.cc:654] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:31.769932 5251 log.cc:824] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:31.770738 5251 tablet_bootstrap.cc:492] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: No bootstrap required, opened a new log
I20260812 05:19:31.770810 5251 ts_tablet_manager.cc:1397] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:31.771072 5252 raft_consensus.cc:348] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:31.771250 5252 raft_consensus.cc:374] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:31.771297 5252 raft_consensus.cc:729] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8545f49f92a84b7c9d1dfa935207c43d, State: Initialized, Role: FOLLOWER
I20260812 05:19:31.771442 5252 consensus_queue.cc:260] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:31.771775 5252 ts_tablet_manager.cc:1428] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:19:31.771915 5090 heartbeater.cc:503] Master 127.18.166.254:42795 was elected leader, sending a full tablet report...
I20260812 05:19:31.772229 5255 tablet_bootstrap.cc:492] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: Bootstrap starting.
I20260812 05:19:31.773005 5255 tablet_bootstrap.cc:654] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:31.772934 5251 raft_consensus.cc:348] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:31.773099 5251 raft_consensus.cc:374] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:31.773149 5251 raft_consensus.cc:729] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ed81e3b51c94bdc9dd03b5e640dfa01, State: Initialized, Role: FOLLOWER
I20260812 05:19:31.773306 5251 consensus_queue.cc:260] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:31.773660 5251 ts_tablet_manager.cc:1428] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 05:19:31.773648 5255 log.cc:824] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:31.773685 5233 heartbeater.cc:503] Master 127.18.166.254:42795 was elected leader, sending a full tablet report...
I20260812 05:19:31.774372 5255 tablet_bootstrap.cc:492] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: No bootstrap required, opened a new log
I20260812 05:19:31.774451 5255 ts_tablet_manager.cc:1397] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:19:31.775803 5255 raft_consensus.cc:348] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:31.775928 5255 raft_consensus.cc:374] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:31.775949 5255 raft_consensus.cc:729] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a881ccf1a74183a3f51934a7a3afa1, State: Initialized, Role: FOLLOWER
I20260812 05:19:31.776074 5255 consensus_queue.cc:260] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:31.776365 5255 ts_tablet_manager.cc:1428] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:19:31.776397 4947 heartbeater.cc:503] Master 127.18.166.254:42795 was elected leader, sending a full tablet report...
W20260812 05:19:31.817472 5091 tablet.cc:2369] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:19:31.887863 4948 tablet.cc:2369] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:19:31.979779 5235 tablet.cc:2369] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:19:32.048305 5257 raft_consensus.cc:482] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:32.048453 5257 raft_consensus.cc:504] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:32.048918 5257 leader_election.cc:288] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28a881ccf1a74183a3f51934a7a3afa1 (127.18.166.193:43809), 8545f49f92a84b7c9d1dfa935207c43d (127.18.166.194:43383)
I20260812 05:19:32.052011 5248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.049316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:48767 (local address 127.18.166.193:43809)
0812 05:19:32.049396 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.049398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.049431 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:19:32.049463 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.049464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.049482 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.049554 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:32.049733 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.050250 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.051241 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.051625 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.051716 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.051726 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.051892 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.051931 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 05:19:32.052011 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.049153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:43809 (local address 127.18.166.195:48767)
0812 05:19:32.049228 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.049245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.049324 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.049559 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.049560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.049562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:32.049650 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.049653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.050357 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.050358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.051130 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.051132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.051713 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.051714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.051811 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.051814 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.051883 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.051932 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 05:19:32.052137 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.049267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:59475 (local address 127.18.166.194:43383)
0812 05:19:32.049323 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.049326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.049624 (+ 298us) server_negotiation.cc:407] Connection header received
0812 05:19:32.049705 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.049706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.049718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.049804 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:32.049976 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.050437 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.051394 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.051777 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.051861 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.051869 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.052030 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.052071 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:19:32.052150 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.049154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:43383 (local address 127.18.166.195:59475)
0812 05:19:32.049544 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.049553 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.049636 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.049815 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.049816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.049817 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:32.049900 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.049902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.050528 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.050529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.051289 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.051291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.051862 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.051863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.051940 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.051944 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.052022 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.052062 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":185,"thread_start_us":139,"threads_started":1}
I20260812 05:19:32.052690 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94574f5800a44311b7ad18c2ab49a699" candidate_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8545f49f92a84b7c9d1dfa935207c43d" is_pre_election: true
I20260812 05:19:32.052917 5040 raft_consensus.cc:2454] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ed81e3b51c94bdc9dd03b5e640dfa01 in term 0.
I20260812 05:19:32.053237 5117 leader_election.cc:302] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ed81e3b51c94bdc9dd03b5e640dfa01, 8545f49f92a84b7c9d1dfa935207c43d; no voters:
I20260812 05:19:32.053341 4898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94574f5800a44311b7ad18c2ab49a699" candidate_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a881ccf1a74183a3f51934a7a3afa1" is_pre_election: true
I20260812 05:19:32.053395 5257 raft_consensus.cc:2781] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:19:32.053474 5257 raft_consensus.cc:482] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:19:32.053509 5257 raft_consensus.cc:3037] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:32.053531 4898 raft_consensus.cc:2454] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ed81e3b51c94bdc9dd03b5e640dfa01 in term 0.
I20260812 05:19:32.054271 5257 raft_consensus.cc:504] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:32.054426 5257 leader_election.cc:288] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [CANDIDATE]: Term 1 election: Requested vote from peers 28a881ccf1a74183a3f51934a7a3afa1 (127.18.166.193:43809), 8545f49f92a84b7c9d1dfa935207c43d (127.18.166.194:43383)
I20260812 05:19:32.054631 4898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94574f5800a44311b7ad18c2ab49a699" candidate_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a881ccf1a74183a3f51934a7a3afa1"
I20260812 05:19:32.054694 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94574f5800a44311b7ad18c2ab49a699" candidate_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8545f49f92a84b7c9d1dfa935207c43d"
I20260812 05:19:32.054754 4898 raft_consensus.cc:3037] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:32.054764 5040 raft_consensus.cc:3037] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:32.055502 5040 raft_consensus.cc:2454] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ed81e3b51c94bdc9dd03b5e640dfa01 in term 1.
I20260812 05:19:32.055675 5117 leader_election.cc:302] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ed81e3b51c94bdc9dd03b5e640dfa01, 8545f49f92a84b7c9d1dfa935207c43d; no voters:
I20260812 05:19:32.055713 4898 raft_consensus.cc:2454] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ed81e3b51c94bdc9dd03b5e640dfa01 in term 1.
I20260812 05:19:32.055786 5257 raft_consensus.cc:2781] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:32.055960 5257 raft_consensus.cc:686] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 1 LEADER]: Becoming Leader. State: Replica: 7ed81e3b51c94bdc9dd03b5e640dfa01, State: Running, Role: LEADER
I20260812 05:19:32.056084 5257 consensus_queue.cc:237] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } }
I20260812 05:19:32.057017 4742 catalog_manager.cc:5495] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ed81e3b51c94bdc9dd03b5e640dfa01 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ed81e3b51c94bdc9dd03b5e640dfa01" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 36697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 } health_report { overall_health: UNKNOWN } } }
I20260812 05:19:32.115213 5250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.112696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.18.166.195:36697)
0812 05:19:32.112747 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.112749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.112759 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:32.112836 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.112837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.112856 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.112930 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:19:32.113095 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.113619 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.114377 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.114571 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.114649 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.114849 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:19:32.114951 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:19:32.115046 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.115072 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.115095 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:19:32.117110 4898 raft_consensus.cc:1261] T 94574f5800a44311b7ad18c2ab49a699 P 28a881ccf1a74183a3f51934a7a3afa1 [term 1 FOLLOWER]: Refusing update from remote peer 7ed81e3b51c94bdc9dd03b5e640dfa01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:32.117118 5040 raft_consensus.cc:1261] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 1 FOLLOWER]: Refusing update from remote peer 7ed81e3b51c94bdc9dd03b5e640dfa01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:32.117548 5261 consensus_queue.cc:1035] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8545f49f92a84b7c9d1dfa935207c43d" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 43383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:32.117702 5257 consensus_queue.cc:1035] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a881ccf1a74183a3f51934a7a3afa1" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 43809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:32.120723 5267 mvcc.cc:205] Tried to move back new op lower bound from 7317553037788598272 to 7317553037542436864. Current Snapshot: MvccSnapshot[applied={T|T < 7317553037788598272 or (T in {7317553037788598272})}]
I20260812 05:19:32.122817 4742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:32.122906 4742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:32.123684 4742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb90267a27e74b408ed3287073fb63bf: Sending DeleteTablet for 3 replicas of tablet 94574f5800a44311b7ad18c2ab49a699
I20260812 05:19:32.124011 5164 tablet_service.cc:1514] Processing DeleteTablet for tablet 94574f5800a44311b7ad18c2ab49a699 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:32 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:40560
I20260812 05:19:32.124042 4877 tablet_service.cc:1514] Processing DeleteTablet for tablet 94574f5800a44311b7ad18c2ab49a699 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:32 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:42828
I20260812 05:19:32.124262 5019 tablet_service.cc:1514] Processing DeleteTablet for tablet 94574f5800a44311b7ad18c2ab49a699 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:32 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:53166
I20260812 05:19:32.124493 5277 tablet_replica.cc:330] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: stopping tablet replica
I20260812 05:19:32.124619 5277 raft_consensus.cc:2229] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:32.124727 5277 raft_consensus.cc:2258] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:32.124742 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 4808
I20260812 05:19:32.129627 5277 ts_tablet_manager.cc:1905] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:32.131659 5277 ts_tablet_manager.cc:1918] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:32.131783 5277 log.cc:1196] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/94574f5800a44311b7ad18c2ab49a699
I20260812 05:19:32.132040 5277 ts_tablet_manager.cc:1939] T 94574f5800a44311b7ad18c2ab49a699 P 8545f49f92a84b7c9d1dfa935207c43d: Deleting consensus metadata
I20260812 05:19:32.132503 4728 catalog_manager.cc:4841] TS 8545f49f92a84b7c9d1dfa935207c43d (127.18.166.194:43383): tablet 94574f5800a44311b7ad18c2ab49a699 (table test-table [id=cf367db2974b4627bfc0c689fe06d28e]) successfully deleted
W20260812 05:19:32.133726 4725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:43809 (error 108)
I20260812 05:19:32.134027 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 4952
I20260812 05:19:32.137545 5275 tablet_replica.cc:330] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: stopping tablet replica
I20260812 05:19:32.137728 5275 raft_consensus.cc:2229] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:19:32.137939 5275 raft_consensus.cc:2258] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:32.138578 5275 ts_tablet_manager.cc:1905] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:32.140283 5275 ts_tablet_manager.cc:1918] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:32.140381 5275 log.cc:1196] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/94574f5800a44311b7ad18c2ab49a699
I20260812 05:19:32.140637 5275 ts_tablet_manager.cc:1939] T 94574f5800a44311b7ad18c2ab49a699 P 7ed81e3b51c94bdc9dd03b5e640dfa01: Deleting consensus metadata
I20260812 05:19:32.141116 4726 catalog_manager.cc:4841] TS 7ed81e3b51c94bdc9dd03b5e640dfa01 (127.18.166.195:36697): tablet 94574f5800a44311b7ad18c2ab49a699 (table test-table [id=cf367db2974b4627bfc0c689fe06d28e]) successfully deleted
I20260812 05:19:32.141868 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.141720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:43809 (local address 127.0.0.1:42836)
0812 05:19:32.141781 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.141828 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 05:19:32.142027 4725 catalog_manager.cc:4584] TS 28a881ccf1a74183a3f51934a7a3afa1 (127.18.166.193:43809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94574f5800a44311b7ad18c2ab49a699: Network error: Client connection negotiation failed: client connection to 127.18.166.193:43809: connect: Connection refused (error 111)
I20260812 05:19:32.149698 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5095
I20260812 05:19:32.155897 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 4708
2026-08-12T05:19:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2298 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T05:19:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:32Z Disabled control of system clock
I20260812 05:19:32.185036 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34409
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:32.267197 5282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:32.267439 5282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:32.267462 5282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:32.268564 5282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:32.268607 5282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:32.268620 5282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:32.268632 5282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:32.268644 5282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:32.269954 5282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:34409
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:32.270154 5282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:32.270392 5282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:32.275271 5287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.275312 5288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.275264 5290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:32.275606 5282 server_base.cc:1034] running on GCE node
I20260812 05:19:32.275874 5282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:32.276136 5282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:32.277339 5282 hybrid_clock.cc:648] HybridClock initialized: now 1786511972277330 us; error 38 us; skew 500 ppm
I20260812 05:19:32.278738 5282 webserver.cc:460] Webserver started at http://127.18.166.254:38573/ using document root <none> and password file <none>
I20260812 05:19:32.279006 5282 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:32.279086 5282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:32.279270 5282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:32.280359 5282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "d2d119f7af944d19b23786354d9367b2"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "baea200671de6bfd7b16153db1ef47a5"
server_key_iv: "6451bc492425469c5cad59aa970b84c4"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.280730 5282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "d2d119f7af944d19b23786354d9367b2"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "baea200671de6bfd7b16153db1ef47a5"
server_key_iv: "6451bc492425469c5cad59aa970b84c4"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.282158 5282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:32.282941 5296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.283159 5282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:32.283268 5282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "d2d119f7af944d19b23786354d9367b2"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "baea200671de6bfd7b16153db1ef47a5"
server_key_iv: "6451bc492425469c5cad59aa970b84c4"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.283347 5282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:32.292956 5282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:32.293314 5282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:32.293543 5282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:32.297698 5282 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:34409
I20260812 05:19:32.297737 5351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:34409 every 8 connection(s)
I20260812 05:19:32.298100 5282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:32.299108 5352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:32.299492 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5282
I20260812 05:19:32.299646 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:32.299885 19099 external_mini_cluster.cc:1428] Setting key 90c00a2c5bf441d7513c3f179bc56d8f
I20260812 05:19:32.302475 5352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2: Bootstrap starting.
I20260812 05:19:32.303187 5352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:32.303521 5352 log.cc:824] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:32.304430 5352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2: No bootstrap required, opened a new log
I20260812 05:19:32.306676 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.300486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.18.166.254:34409)
0812 05:19:32.300712 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.300721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.300766 (+ 45us) server_negotiation.cc:407] Connection header received
0812 05:19:32.301308 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.301316 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.301395 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.301567 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:32.301726 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.303084 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.304233 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.304853 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.304993 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.305823 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:32.305827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:32.305828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:32.306039 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:32.306087 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:32.306103 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:32.306183 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.306309 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.306414 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260812 05:19:32.306874 5352 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2d119f7af944d19b23786354d9367b2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34409 } }
I20260812 05:19:32.307024 5352 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:32.307070 5352 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d119f7af944d19b23786354d9367b2, State: Initialized, Role: FOLLOWER
I20260812 05:19:32.307233 5352 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2d119f7af944d19b23786354d9367b2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34409 } }
I20260812 05:19:32.307333 5352 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:32.307379 5352 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:32.307430 5352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:32.308012 5352 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2d119f7af944d19b23786354d9367b2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34409 } }
I20260812 05:19:32.308102 5352 leader_election.cc:302] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2d119f7af944d19b23786354d9367b2; no voters:
I20260812 05:19:32.308308 5352 leader_election.cc:288] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:32.308385 5357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:32.308545 5357 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [term 1 LEADER]: Becoming Leader. State: Replica: d2d119f7af944d19b23786354d9367b2, State: Running, Role: LEADER
I20260812 05:19:32.308590 5352 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:32.308629 5357 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2d119f7af944d19b23786354d9367b2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34409 } }
I20260812 05:19:32.309070 5358 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2d119f7af944d19b23786354d9367b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2d119f7af944d19b23786354d9367b2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34409 } } }
I20260812 05:19:32.309198 5358 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:32.309367 5359 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2d119f7af944d19b23786354d9367b2. Latest consensus state: current_term: 1 leader_uuid: "d2d119f7af944d19b23786354d9367b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2d119f7af944d19b23786354d9367b2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34409 } } }
I20260812 05:19:32.309478 5359 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:32.309731 5366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:32.310237 5366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:32.311930 5366 catalog_manager.cc:1347] Generated new cluster ID: e3a14955ffba40aebbd45452d4434366
I20260812 05:19:32.311990 5366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:32.328020 5366 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:32.328791 5366 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:32.335276 5366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2: Generated new TSK 0
I20260812 05:19:32.335496 5366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:32.344250 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34409
--builtin_ntp_servers=127.18.166.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:32.425956 5378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:32.426163 5378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:32.426185 5378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:32.427337 5378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:32.427382 5378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:32.427408 5378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:32.428758 5378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:32.428997 5378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:32.429351 5378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:32.435429 5384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.435443 5383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.435436 5387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:32.436198 5378 server_base.cc:1034] running on GCE node
I20260812 05:19:32.436429 5378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:32.436695 5378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:32.437850 5378 hybrid_clock.cc:648] HybridClock initialized: now 1786511972437834 us; error 32 us; skew 500 ppm
I20260812 05:19:32.439257 5378 webserver.cc:460] Webserver started at http://127.18.166.193:40671/ using document root <none> and password file <none>
I20260812 05:19:32.439512 5378 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:32.439581 5378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:32.439774 5378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:32.441021 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "d00e5892f9354fa7ae812744f2e97045"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "c62ac96bed37c67d6c0d6a070cbdeefd"
server_key_iv: "13886be953db092b85024052c71b0ede"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.441512 5378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "d00e5892f9354fa7ae812744f2e97045"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "c62ac96bed37c67d6c0d6a070cbdeefd"
server_key_iv: "13886be953db092b85024052c71b0ede"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.442924 5378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:32.443758 5393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.443928 5378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:32.444024 5378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "d00e5892f9354fa7ae812744f2e97045"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "c62ac96bed37c67d6c0d6a070cbdeefd"
server_key_iv: "13886be953db092b85024052c71b0ede"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.444089 5378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:32.452975 5378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:32.453228 5378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:32.453330 5378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:32.453570 5378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:32.453856 5378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:32.453887 5378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.453908 5378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:32.453922 5378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.459486 5378 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:39951
I20260812 05:19:32.459560 5513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:39951 every 8 connection(s)
I20260812 05:19:32.459906 5378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:19:32.465291 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.460698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:34241 (local address 127.18.166.254:34409)
0812 05:19:32.460759 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.460761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.461022 (+ 261us) server_negotiation.cc:407] Connection header received
0812 05:19:32.461611 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.461613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.461626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.461690 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:32.462362 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.462883 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.463873 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.464082 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.464161 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.464986 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:32.464987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:32.464987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:32.464992 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:32.465008 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:32.465012 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:32.465080 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.465174 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.465205 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:19:32.465565 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.460674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34409 (local address 127.18.166.193:34241)
0812 05:19:32.460900 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.460933 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.461507 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.461764 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.461768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.461775 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:32.462268 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.462272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.462985 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.462986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.463778 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.463780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.464172 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.464174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.464316 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.464843 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:32.464844 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:32.464872 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:32.464894 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:32.465080 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:32.465087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:32.465091 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.465176 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.465303 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260812 05:19:32.466125 5514 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34409
I20260812 05:19:32.466241 5514 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:32.466451 5514 heartbeater.cc:511] Master 127.18.166.254:34409 requested a full tablet report, sending...
I20260812 05:19:32.466980 5315 ts_manager.cc:194] Registered new tserver with Master: d00e5892f9354fa7ae812744f2e97045 (127.18.166.193:39951)
I20260812 05:19:32.467550 5315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.193:34241
I20260812 05:19:32.469763 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5378
I20260812 05:19:32.469847 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:19:32.470021 19099 external_mini_cluster.cc:1428] Setting key ec00e341c71dec574627402d2697c4d7
I20260812 05:19:32.470903 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34409
--builtin_ntp_servers=127.18.166.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:32.552966 5518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:32.553196 5518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:32.553233 5518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:32.554397 5518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:32.554461 5518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:32.554505 5518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:32.555850 5518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:32.556094 5518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:32.556380 5518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:32.561393 5524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.561409 5526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.561431 5523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:32.561503 5518 server_base.cc:1034] running on GCE node
I20260812 05:19:32.561856 5518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:32.562146 5518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:32.563309 5518 hybrid_clock.cc:648] HybridClock initialized: now 1786511972563295 us; error 46 us; skew 500 ppm
I20260812 05:19:32.564524 5518 webserver.cc:460] Webserver started at http://127.18.166.194:32907/ using document root <none> and password file <none>
I20260812 05:19:32.564780 5518 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:32.564850 5518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:32.565016 5518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:32.566164 5518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "1891e1c389f62defb21d516c99d065f1"
server_key_iv: "bde3d06bae3653ec30e6a4623ddae206"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.566524 5518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "1891e1c389f62defb21d516c99d065f1"
server_key_iv: "bde3d06bae3653ec30e6a4623ddae206"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.567839 5518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:19:32.568851 5532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.569136 5518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:32.569264 5518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "1891e1c389f62defb21d516c99d065f1"
server_key_iv: "bde3d06bae3653ec30e6a4623ddae206"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.569360 5518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:32.595332 5518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:32.595746 5518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:32.595942 5518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:32.596230 5518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:32.596597 5518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:32.596662 5518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.596706 5518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:32.596763 5518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.602852 5518 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:35345
I20260812 05:19:32.602900 5651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:35345 every 8 connection(s)
I20260812 05:19:32.603252 5518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:19:32.606141 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5518
I20260812 05:19:32.606271 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:19:32.606513 19099 external_mini_cluster.cc:1428] Setting key 32bbcbe9a3dc07c598377b46b3fa4fdb
I20260812 05:19:32.607549 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34409
--builtin_ntp_servers=127.18.166.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:19:32.609642 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.604061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:38291 (local address 127.18.166.254:34409)
0812 05:19:32.604134 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.604137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.604432 (+ 295us) server_negotiation.cc:407] Connection header received
0812 05:19:32.604920 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.604926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.604937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.604995 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:32.605801 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.606411 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.607837 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.608077 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.608145 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.609235 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:32.609237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:32.609237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:32.609243 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:32.609262 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:32.609268 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:32.609329 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.609492 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.609523 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:19:32.609840 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.604046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34409 (local address 127.18.166.194:38291)
0812 05:19:32.604298 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.604337 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.604830 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.605071 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.605074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.605082 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:32.605695 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.605700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.606508 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.606509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.607765 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.607768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.608191 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.608192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.608366 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.609054 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:32.609055 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:32.609100 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:32.609134 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:32.609359 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:32.609365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:32.609368 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.609471 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.609571 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
I20260812 05:19:32.610302 5652 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34409
I20260812 05:19:32.610394 5652 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:32.610600 5652 heartbeater.cc:511] Master 127.18.166.254:34409 requested a full tablet report, sending...
I20260812 05:19:32.610999 5315 ts_manager.cc:194] Registered new tserver with Master: b8f2506e2fbb498690d5ffdcd4b3e4ee (127.18.166.194:35345)
I20260812 05:19:32.611537 5315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.194:38291
W20260812 05:19:32.691220 5655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:32.691428 5655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:32.691450 5655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:32.693229 5655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:32.693272 5655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:32.693298 5655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:32.694653 5655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:34409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:32.694842 5655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:32.695106 5655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:32.700048 5662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.700059 5664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:32.700067 5661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:32.700413 5655 server_base.cc:1034] running on GCE node
I20260812 05:19:32.700649 5655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:32.700913 5655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:32.702072 5655 hybrid_clock.cc:648] HybridClock initialized: now 1786511972702072 us; error 43 us; skew 500 ppm
I20260812 05:19:32.703332 5655 webserver.cc:460] Webserver started at http://127.18.166.195:38333/ using document root <none> and password file <none>
I20260812 05:19:32.703591 5655 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:32.703662 5655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:32.703816 5655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:32.704911 5655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "8dccf67a1dce4f82aa021f0d972915c2"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "70fb8f66a9ef768e097824843aeb3912"
server_key_iv: "fed91d6ccb5ad4b69036480cd1d2da5c"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.705281 5655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "8dccf67a1dce4f82aa021f0d972915c2"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "70fb8f66a9ef768e097824843aeb3912"
server_key_iv: "fed91d6ccb5ad4b69036480cd1d2da5c"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.706636 5655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:32.707358 5670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.707526 5655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:32.707633 5655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "8dccf67a1dce4f82aa021f0d972915c2"
format_stamp: "Formatted at 2026-08-12 05:19:32 on dist-test-slave-srgv"
server_key: "70fb8f66a9ef768e097824843aeb3912"
server_key_iv: "fed91d6ccb5ad4b69036480cd1d2da5c"
server_key_version: "encryptionkey@0"
I20260812 05:19:32.707715 5655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:32.729816 5655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:32.730187 5655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:32.730353 5655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:32.730603 5655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:32.730942 5655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:32.730995 5655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.731041 5655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:32.731074 5655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:32.736732 5655 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:32785
I20260812 05:19:32.736809 5790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:32785 every 8 connection(s)
I20260812 05:19:32.737166 5655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:19:32.743084 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.738011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:46065 (local address 127.18.166.254:34409)
0812 05:19:32.738072 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.738074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.738357 (+ 283us) server_negotiation.cc:407] Connection header received
0812 05:19:32.738911 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.738912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.738925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.738990 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:32.739853 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.740707 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.741713 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.741922 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.742001 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.742788 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:32.742789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:32.742790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:32.742795 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:32.742810 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:32.742815 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:32.742881 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.742976 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.743001 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:19:32.743311 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.737992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:34409 (local address 127.18.166.195:46065)
0812 05:19:32.738190 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.738226 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.738800 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.739067 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.739070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.739078 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:32.739747 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.739753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.740804 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.740805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.741610 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.741612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.742008 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.742009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.742144 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.742640 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:32.742640 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:32.742667 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:32.742689 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:32.742881 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:32.742887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:32.742890 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.742977 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.743077 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260812 05:19:32.743803 5791 heartbeater.cc:347] Connected to a master server at 127.18.166.254:34409
I20260812 05:19:32.743815 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5655
I20260812 05:19:32.743882 5791 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:32.743952 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:19:32.744108 5791 heartbeater.cc:511] Master 127.18.166.254:34409 requested a full tablet report, sending...
I20260812 05:19:32.744174 19099 external_mini_cluster.cc:1428] Setting key 5ad1a54c83c55ca423520eae10c11338
I20260812 05:19:32.744536 5315 ts_manager.cc:194] Registered new tserver with Master: 8dccf67a1dce4f82aa021f0d972915c2 (127.18.166.195:32785)
I20260812 05:19:32.745060 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:19:32.745048 5315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.195:46065
I20260812 05:19:32.749089 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.746488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.18.166.254:34409)
0812 05:19:32.746522 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.746524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.746648 (+ 124us) server_negotiation.cc:407] Connection header received
0812 05:19:32.746708 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.746709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.746717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.746755 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:32.746871 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.747385 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.748073 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.748249 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.748300 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.748932 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:32.748933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:32.748933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:32.748937 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:32.748946 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:32.748949 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:32.748986 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.749022 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.749037 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 05:19:32.750707 5315 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:19:32.752214 5315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:19:32.758615 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.754592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.18.166.194:35345)
0812 05:19:32.754923 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.754928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.754939 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:19:32.754980 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.754982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.755025 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.755313 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:32.755381 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.756502 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.757742 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.758285 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.758375 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.758468 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.758502 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.758539 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":87,"threads_started":1}
I20260812 05:19:32.758603 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.754407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:35345 (local address 127.0.0.1:36656)
0812 05:19:32.754571 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.754607 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.754671 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.755142 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.755145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.755148 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:32.755248 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.755250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.756600 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.756601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.757649 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.757651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.758373 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.758374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.758433 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.758436 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.758474 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.758515 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":71,"spinlock_wait_cycles":768,"thread_start_us":83,"threads_started":1}
I20260812 05:19:32.758700 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.754407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:32785 (local address 127.0.0.1:40172)
0812 05:19:32.754732 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.754743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.754797 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.755158 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.755158 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.755160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:32.755241 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.755243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.756618 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.756619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.757618 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.757621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.758502 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.758503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.758562 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.758565 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.758596 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.758628 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":27,"thread_start_us":59,"threads_started":1}
I20260812 05:19:32.758888 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.754832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.18.166.193:39951)
0812 05:19:32.755004 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.755008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.755018 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:32.755054 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.755056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.755070 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.755137 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:32.755324 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.756045 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.757388 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.758158 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.758566 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.758662 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.758749 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.758809 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260812 05:19:32.759043 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.754491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.18.166.195:32785)
0812 05:19:32.754882 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:32.754887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:32.754897 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:32.754934 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:32.754936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:32.754953 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:32.755036 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:32.755431 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.756511 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.757712 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.758416 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.758809 (+ 393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.758894 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:32.758920 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 05:19:32.758964 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":106,"threads_started":1}
I20260812 05:19:32.759292 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:32.754605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:39951 (local address 127.0.0.1:48182)
0812 05:19:32.754784 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:32.754794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:32.754839 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:32.755149 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:32.755149 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:32.755152 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:32.755238 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.755240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.756135 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.756136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.757300 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:32.757302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:32.758261 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:32.758262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:32.758342 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:32.758357 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:19:32.758406 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 05:19:32.758466 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":5,"thread_start_us":74,"threads_started":1}
I20260812 05:19:32.759613 5583 tablet_service.cc:1467] Processing CreateTablet for tablet d01e532593614949b3a712d484b99893 (DEFAULT_TABLE table=test-table [id=09c4fc9ab0fb4356b45c4be1b081344a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:32.759960 5583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d01e532593614949b3a712d484b99893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:32.759903 5445 tablet_service.cc:1467] Processing CreateTablet for tablet d01e532593614949b3a712d484b99893 (DEFAULT_TABLE table=test-table [id=09c4fc9ab0fb4356b45c4be1b081344a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:32.760210 5445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d01e532593614949b3a712d484b99893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:32.761844 5724 tablet_service.cc:1467] Processing CreateTablet for tablet d01e532593614949b3a712d484b99893 (DEFAULT_TABLE table=test-table [id=09c4fc9ab0fb4356b45c4be1b081344a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:32.762184 5724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d01e532593614949b3a712d484b99893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:32.762669 5808 tablet_bootstrap.cc:492] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Bootstrap starting.
I20260812 05:19:32.763093 5809 tablet_bootstrap.cc:492] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: Bootstrap starting.
I20260812 05:19:32.763489 5808 tablet_bootstrap.cc:654] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:32.763763 5809 tablet_bootstrap.cc:654] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:32.763820 5808 log.cc:824] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:32.764108 5809 log.cc:824] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:32.764612 5808 tablet_bootstrap.cc:492] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: No bootstrap required, opened a new log
I20260812 05:19:32.764703 5808 ts_tablet_manager.cc:1397] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:19:32.765249 5812 tablet_bootstrap.cc:492] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Bootstrap starting.
I20260812 05:19:32.766002 5809 tablet_bootstrap.cc:492] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: No bootstrap required, opened a new log
I20260812 05:19:32.766079 5809 ts_tablet_manager.cc:1397] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:19:32.767349 5808 raft_consensus.cc:348] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:32.767521 5808 raft_consensus.cc:374] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:32.767558 5808 raft_consensus.cc:729] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f2506e2fbb498690d5ffdcd4b3e4ee, State: Initialized, Role: FOLLOWER
I20260812 05:19:32.767676 5808 consensus_queue.cc:260] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:32.767949 5808 ts_tablet_manager.cc:1428] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 05:19:32.767998 5652 heartbeater.cc:503] Master 127.18.166.254:34409 was elected leader, sending a full tablet report...
I20260812 05:19:32.768553 5809 raft_consensus.cc:348] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:32.768715 5809 raft_consensus.cc:374] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:32.768757 5809 raft_consensus.cc:729] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00e5892f9354fa7ae812744f2e97045, State: Initialized, Role: FOLLOWER
I20260812 05:19:32.768891 5809 consensus_queue.cc:260] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:32.769172 5809 ts_tablet_manager.cc:1428] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:19:32.769200 5514 heartbeater.cc:503] Master 127.18.166.254:34409 was elected leader, sending a full tablet report...
I20260812 05:19:32.769703 5812 tablet_bootstrap.cc:654] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:32.770033 5812 log.cc:824] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:32.770699 5812 tablet_bootstrap.cc:492] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: No bootstrap required, opened a new log
I20260812 05:19:32.770753 5812 ts_tablet_manager.cc:1397] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 05:19:32.772471 5812 raft_consensus.cc:348] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:32.772589 5812 raft_consensus.cc:374] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:32.772610 5812 raft_consensus.cc:729] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dccf67a1dce4f82aa021f0d972915c2, State: Initialized, Role: FOLLOWER
I20260812 05:19:32.772713 5812 consensus_queue.cc:260] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:32.772969 5812 ts_tablet_manager.cc:1428] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:32.773005 5791 heartbeater.cc:503] Master 127.18.166.254:34409 was elected leader, sending a full tablet report...
W20260812 05:19:32.853842 5653 tablet.cc:2369] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:19:32.960638 5516 tablet.cc:2369] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:19:32.987741 5792 tablet.cc:2369] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:19:33.075820 5815 raft_consensus.cc:482] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:33.075999 5815 raft_consensus.cc:504] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:33.076602 5815 leader_election.cc:288] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dccf67a1dce4f82aa021f0d972915c2 (127.18.166.195:32785), b8f2506e2fbb498690d5ffdcd4b3e4ee (127.18.166.194:35345)
I20260812 05:19:33.079726 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:33.076882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:45009 (local address 127.18.166.195:32785)
0812 05:19:33.076933 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:33.076936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:33.077056 (+ 120us) server_negotiation.cc:407] Connection header received
0812 05:19:33.077148 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:33.077149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:33.077162 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:33.077232 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:33.077448 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.077907 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.078999 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.079367 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.079453 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:33.079460 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:33.079618 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 05:19:33.079657 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:19:33.079751 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:33.076807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:32785 (local address 127.18.166.193:45009)
0812 05:19:33.076971 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:33.076989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:33.077073 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:33.077236 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:33.077237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:33.077239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:33.077354 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:33.077356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.077999 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.078000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:33.078891 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:33.078893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.079453 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.079453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:33.079526 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:33.079536 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 05:19:33.079599 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 05:19:33.079655 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":69,"thread_start_us":71,"threads_started":1}
I20260812 05:19:33.079744 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:33.076920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:45379 (local address 127.18.166.194:35345)
0812 05:19:33.076980 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:33.076982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:33.077069 (+ 87us) server_negotiation.cc:407] Connection header received
0812 05:19:33.077146 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:33.077147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:33.077161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:33.077239 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:33.077440 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.077917 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.079001 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.079355 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.079440 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:33.079448 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:33.079600 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 05:19:33.079643 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:19:33.079754 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:33.076833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:35345 (local address 127.18.166.193:45379)
0812 05:19:33.076978 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:33.076991 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:33.077086 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:33.077244 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:33.077245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:33.077247 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:33.077340 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:33.077342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.078013 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.078015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:33.078885 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:33.078887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.079438 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.079440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:33.079521 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:33.079525 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:33.079596 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:19:33.079647 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":24,"thread_start_us":65,"threads_started":1}
I20260812 05:19:33.080227 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01e532593614949b3a712d484b99893" candidate_uuid: "d00e5892f9354fa7ae812744f2e97045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dccf67a1dce4f82aa021f0d972915c2" is_pre_election: true
I20260812 05:19:33.080394 5744 raft_consensus.cc:2454] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d00e5892f9354fa7ae812744f2e97045 in term 0.
I20260812 05:19:33.080690 5397 leader_election.cc:302] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dccf67a1dce4f82aa021f0d972915c2, d00e5892f9354fa7ae812744f2e97045; no voters:
I20260812 05:19:33.080819 5815 raft_consensus.cc:2781] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:19:33.080883 5815 raft_consensus.cc:482] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:19:33.080910 5815 raft_consensus.cc:3037] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:33.080822 5603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01e532593614949b3a712d484b99893" candidate_uuid: "d00e5892f9354fa7ae812744f2e97045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" is_pre_election: true
I20260812 05:19:33.080991 5603 raft_consensus.cc:2454] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d00e5892f9354fa7ae812744f2e97045 in term 0.
I20260812 05:19:33.081763 5815 raft_consensus.cc:504] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:33.081925 5815 leader_election.cc:288] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [CANDIDATE]: Term 1 election: Requested vote from peers 8dccf67a1dce4f82aa021f0d972915c2 (127.18.166.195:32785), b8f2506e2fbb498690d5ffdcd4b3e4ee (127.18.166.194:35345)
I20260812 05:19:33.082073 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01e532593614949b3a712d484b99893" candidate_uuid: "d00e5892f9354fa7ae812744f2e97045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dccf67a1dce4f82aa021f0d972915c2"
I20260812 05:19:33.082079 5603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d01e532593614949b3a712d484b99893" candidate_uuid: "d00e5892f9354fa7ae812744f2e97045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee"
I20260812 05:19:33.082157 5603 raft_consensus.cc:3037] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:33.082159 5744 raft_consensus.cc:3037] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:33.083076 5744 raft_consensus.cc:2454] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d00e5892f9354fa7ae812744f2e97045 in term 1.
I20260812 05:19:33.083076 5603 raft_consensus.cc:2454] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d00e5892f9354fa7ae812744f2e97045 in term 1.
I20260812 05:19:33.083244 5396 leader_election.cc:302] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8f2506e2fbb498690d5ffdcd4b3e4ee, d00e5892f9354fa7ae812744f2e97045; no voters:
I20260812 05:19:33.083359 5815 raft_consensus.cc:2781] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:33.083549 5815 raft_consensus.cc:686] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [term 1 LEADER]: Becoming Leader. State: Replica: d00e5892f9354fa7ae812744f2e97045, State: Running, Role: LEADER
I20260812 05:19:33.083624 5815 consensus_queue.cc:237] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } }
I20260812 05:19:33.084508 5315 catalog_manager.cc:5495] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 reported cstate change: term changed from 0 to 1, leader changed from <none> to d00e5892f9354fa7ae812744f2e97045 (127.18.166.193). New cstate: current_term: 1 leader_uuid: "d00e5892f9354fa7ae812744f2e97045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d00e5892f9354fa7ae812744f2e97045" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 39951 } health_report { overall_health: HEALTHY } } }
I20260812 05:19:33.114308 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:33.111871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.18.166.193:39951)
0812 05:19:33.111930 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:33.111932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:33.111939 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:19:33.111986 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:33.111986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:33.112006 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:33.112085 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:19:33.112214 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.112710 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.113497 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:33.113713 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:33.113805 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:33.113968 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:19:33.114080 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:19:33.114175 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:33.114195 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 05:19:33.114218 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:19:33.116295 5744 raft_consensus.cc:1261] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 1 FOLLOWER]: Refusing update from remote peer d00e5892f9354fa7ae812744f2e97045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:33.116331 5603 raft_consensus.cc:1261] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 1 FOLLOWER]: Refusing update from remote peer d00e5892f9354fa7ae812744f2e97045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:33.116719 5820 consensus_queue.cc:1035] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8f2506e2fbb498690d5ffdcd4b3e4ee" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:33.116842 5815 consensus_queue.cc:1035] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dccf67a1dce4f82aa021f0d972915c2" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:33.119922 5824 mvcc.cc:205] Tried to move back new op lower bound from 7317553041881264128 to 7317553041751166976. Current Snapshot: MvccSnapshot[applied={T|T < 7317553041881264128 or (T in {7317553041881264128})}]
I20260812 05:19:33.120273 5823 mvcc.cc:205] Tried to move back new op lower bound from 7317553041881264128 to 7317553041751166976. Current Snapshot: MvccSnapshot[applied={T|T < 7317553041881264128 or (T in {7317553041881264128})}]
I20260812 05:19:33.121572 5314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:33.121665 5314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:33.122463 5314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2d119f7af944d19b23786354d9367b2: Sending DeleteTablet for 3 replicas of tablet d01e532593614949b3a712d484b99893
I20260812 05:19:33.122937 5724 tablet_service.cc:1514] Processing DeleteTablet for tablet d01e532593614949b3a712d484b99893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:33 UTC) from {username='slave'} at 127.0.0.1:40172
I20260812 05:19:33.123025 5445 tablet_service.cc:1514] Processing DeleteTablet for tablet d01e532593614949b3a712d484b99893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:33 UTC) from {username='slave'} at 127.0.0.1:48182
I20260812 05:19:33.123042 5583 tablet_service.cc:1514] Processing DeleteTablet for tablet d01e532593614949b3a712d484b99893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:33 UTC) from {username='slave'} at 127.0.0.1:36656
I20260812 05:19:33.123306 5832 tablet_replica.cc:330] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: stopping tablet replica
I20260812 05:19:33.123405 5833 tablet_replica.cc:330] T d01e532593614949b3a712d484b99893 P d00e5892f9354fa7ae812744f2e97045: stopping tablet replica
I20260812 05:19:33.123415 5832 raft_consensus.cc:2229] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:33.123432 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5378
I20260812 05:19:33.123523 5832 raft_consensus.cc:2258] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:33.124157 5832 ts_tablet_manager.cc:1905] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:33.125886 5832 ts_tablet_manager.cc:1918] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:33.125959 5832 log.cc:1196] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/d01e532593614949b3a712d484b99893
I20260812 05:19:33.126255 5832 ts_tablet_manager.cc:1939] T d01e532593614949b3a712d484b99893 P 8dccf67a1dce4f82aa021f0d972915c2: Deleting consensus metadata
I20260812 05:19:33.128749 5300 catalog_manager.cc:4841] TS 8dccf67a1dce4f82aa021f0d972915c2 (127.18.166.195:32785): tablet d01e532593614949b3a712d484b99893 (table test-table [id=09c4fc9ab0fb4356b45c4be1b081344a]) successfully deleted
I20260812 05:19:33.129482 5834 tablet_replica.cc:330] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: stopping tablet replica
I20260812 05:19:33.129623 5834 raft_consensus.cc:2229] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:33.129745 5834 raft_consensus.cc:2258] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:33.130250 5834 ts_tablet_manager.cc:1905] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:33.132397 5834 ts_tablet_manager.cc:1918] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:33.132483 5834 log.cc:1196] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/d01e532593614949b3a712d484b99893
I20260812 05:19:33.132737 5834 ts_tablet_manager.cc:1939] T d01e532593614949b3a712d484b99893 P b8f2506e2fbb498690d5ffdcd4b3e4ee: Deleting consensus metadata
I20260812 05:19:33.133229 5299 catalog_manager.cc:4841] TS b8f2506e2fbb498690d5ffdcd4b3e4ee (127.18.166.194:35345): tablet d01e532593614949b3a712d484b99893 (table test-table [id=09c4fc9ab0fb4356b45c4be1b081344a]) successfully deleted
W20260812 05:19:33.134104 5299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:39951 (error 108)
I20260812 05:19:33.134475 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5518
I20260812 05:19:33.134706 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:33.134545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:39951 (local address 127.0.0.1:48198)
0812 05:19:33.134619 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:33.134663 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:39951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 05:19:33.135555 5299 catalog_manager.cc:4584] TS d00e5892f9354fa7ae812744f2e97045 (127.18.166.193:39951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d01e532593614949b3a712d484b99893: Network error: Client connection negotiation failed: client connection to 127.18.166.193:39951: connect: Connection refused (error 111)
I20260812 05:19:33.141266 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5655
I20260812 05:19:33.147315 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5282
2026-08-12T05:19:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (991 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:33 dist-test-slave-srgv krb5kdc[5839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:33 dist-test-slave-srgv krb5kdc[5839](info): set up 2 sockets
Aug 12 05:19:33 dist-test-slave-srgv krb5kdc[5839](info): commencing operation
krb5kdc: starting...
W20260812 05:19:34.230798 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:19:34.356760 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39357
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:39357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:34.446110 5852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:34.446372 5852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:34.446412 5852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:34.447618 5852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:34.447688 5852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:34.447715 5852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:34.447749 5852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:34.447777 5852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:34.449322 5852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:39357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:34.449640 5852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:34.449882 5852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:34.455044 5860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.455070 5857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.455289 5858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:34.455399 5852 server_base.cc:1034] running on GCE node
I20260812 05:19:34.455642 5852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:34.455982 5852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:34.457165 5852 hybrid_clock.cc:648] HybridClock initialized: now 1786511974457141 us; error 39 us; skew 500 ppm
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:34.460043 5852 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:34.460599 5852 webserver.cc:460] Webserver started at http://127.18.166.254:42859/ using document root <none> and password file <none>
I20260812 05:19:34.460839 5852 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:34.460909 5852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:34.461071 5852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:34.462255 5852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "79c315674600471486b053dc5d1ae460"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "5b243e058bdcdc20e5ebeba8392e5ebb"
server_key_iv: "31fb56f8babc953fce1a96fc6fb2ff51"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.462646 5852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "79c315674600471486b053dc5d1ae460"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "5b243e058bdcdc20e5ebeba8392e5ebb"
server_key_iv: "31fb56f8babc953fce1a96fc6fb2ff51"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.464085 5852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:19:34.464929 5868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.465167 5852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:34.465282 5852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "79c315674600471486b053dc5d1ae460"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "5b243e058bdcdc20e5ebeba8392e5ebb"
server_key_iv: "31fb56f8babc953fce1a96fc6fb2ff51"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.465359 5852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:34.475400 5852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:34.477624 5852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:34.477797 5852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:34.482180 5852 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:39357
I20260812 05:19:34.482224 5922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:39357 every 8 connection(s)
I20260812 05:19:34.482616 5852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:34.483480 5923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:34.486390 5923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460: Bootstrap starting.
I20260812 05:19:34.487048 5923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:34.487391 5923 log.cc:824] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:34.488138 5923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460: No bootstrap required, opened a new log
I20260812 05:19:34.489818 5923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c315674600471486b053dc5d1ae460" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39357 } }
I20260812 05:19:34.489960 5923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:34.489984 5923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c315674600471486b053dc5d1ae460, State: Initialized, Role: FOLLOWER
I20260812 05:19:34.490145 5923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c315674600471486b053dc5d1ae460" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39357 } }
I20260812 05:19:34.490226 5923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:34.490248 5923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:34.490317 5923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:34.490906 5923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c315674600471486b053dc5d1ae460" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39357 } }
I20260812 05:19:34.491017 5923 leader_election.cc:302] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79c315674600471486b053dc5d1ae460; no voters:
I20260812 05:19:34.491196 5923 leader_election.cc:288] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:34.491232 5926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:34.491430 5926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [term 1 LEADER]: Becoming Leader. State: Replica: 79c315674600471486b053dc5d1ae460, State: Running, Role: LEADER
I20260812 05:19:34.491470 5923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:34.491523 5926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c315674600471486b053dc5d1ae460" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39357 } }
I20260812 05:19:34.492143 5928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79c315674600471486b053dc5d1ae460. Latest consensus state: current_term: 1 leader_uuid: "79c315674600471486b053dc5d1ae460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c315674600471486b053dc5d1ae460" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39357 } } }
I20260812 05:19:34.492234 5928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:34.492419 5927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79c315674600471486b053dc5d1ae460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79c315674600471486b053dc5d1ae460" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39357 } } }
I20260812 05:19:34.492558 5927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:34.492697 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5852
I20260812 05:19:34.492818 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:34.493024 19099 external_mini_cluster.cc:1428] Setting key 710e142fa1f6f60acfc1c18213047491
I20260812 05:19:34.493386 5936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:34.494273 5936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:34.496279 5936 catalog_manager.cc:1347] Generated new cluster ID: e3e5cd2be70b4a6fb63ed9f47660366b
I20260812 05:19:34.496348 5936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:34.505074 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:34.493617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.18.166.254:39357)
0812 05:19:34.493804 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:34.493815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:34.493848 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:19:34.493981 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:34.493990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:34.494051 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:34.494211 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:34.495032 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.496168 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.497264 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.497914 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.498013 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:34.500983 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:34.500988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:34.500988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:34.501002 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:34.503235 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:34.503652 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:34.503656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:34.503657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:34.503803 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:34.504013 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:34.504014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:34.504015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:34.504397 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:34.504569 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:34.504667 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 05:19:34.504819 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260812 05:19:34.518759 5936 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:34.519315 5936 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:34.523067 5936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460: Generated new TSK 0
I20260812 05:19:34.523294 5936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:19:34.556195 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39357
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:34.639993 5953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:34.640306 5953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:34.640348 5953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:34.641682 5953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:34.641746 5953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:34.641794 5953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:34.643275 5953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:34.643538 5953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:34.643775 5953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:34.648928 5958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:34.649029 5953 server_base.cc:1034] running on GCE node
W20260812 05:19:34.648931 5962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.649077 5959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:34.649375 5953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:34.649669 5953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:34.650816 5953 hybrid_clock.cc:648] HybridClock initialized: now 1786511974650811 us; error 36 us; skew 500 ppm
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:34.653395 5953 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:19:34.653908 5953 webserver.cc:460] Webserver started at http://127.18.166.193:37905/ using document root <none> and password file <none>
I20260812 05:19:34.654136 5953 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:34.654204 5953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:34.654345 5953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:34.655395 5953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "b6bb4adefbd2465a9d0a3b109b407e47"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "a66b401baad8231cc82492a63aac9ae1"
server_key_iv: "ae72a8f0a0dea274ee32ac0b476eb773"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.655769 5953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "b6bb4adefbd2465a9d0a3b109b407e47"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "a66b401baad8231cc82492a63aac9ae1"
server_key_iv: "ae72a8f0a0dea274ee32ac0b476eb773"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.657166 5953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:19:34.658051 5969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.658249 5953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:19:34.658351 5953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "b6bb4adefbd2465a9d0a3b109b407e47"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "a66b401baad8231cc82492a63aac9ae1"
server_key_iv: "ae72a8f0a0dea274ee32ac0b476eb773"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.658437 5953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:34.667646 5953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:34.669840 5953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:34.669952 5953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:34.670136 5953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:34.670446 5953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:34.670480 5953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.670523 5953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:34.670548 5953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.676431 5953 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:37969
I20260812 05:19:34.676487 6088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:37969 every 8 connection(s)
I20260812 05:19:34.676797 5953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:19:34.681728 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 5953
I20260812 05:19:34.681874 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:19:34.682128 19099 external_mini_cluster.cc:1428] Setting key 8c416a3180f20936e20eb88c1086b0cb
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:34.686949 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:34.677593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:39401 (local address 127.18.166.254:39357)
0812 05:19:34.677657 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:34.677659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:34.677862 (+ 203us) server_negotiation.cc:407] Connection header received
0812 05:19:34.678379 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:34.678380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:34.678398 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:34.678473 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:34.679399 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.679940 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.680885 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.681079 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.681130 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:34.683859 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:34.683861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:34.683861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:34.683868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:34.685506 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:34.685932 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:34.685933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:34.685933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:34.685999 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:34.686284 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:34.686285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:34.686286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:34.686407 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:34.686487 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:34.686768 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 05:19:34.686857 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:19:34.687165 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:34.677556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39357 (local address 127.18.166.193:39401)
0812 05:19:34.677729 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:34.677770 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:34.678245 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:34.678530 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:34.678534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:34.679116 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:34.679307 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:34.679312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.680024 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.680025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:34.680807 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:34.680808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.681154 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.681155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:34.681250 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:34.681835 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:34.681845 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:34.683770 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:34.685607 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:34.685614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:34.685617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:34.685851 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:34.686074 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:34.686075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:34.686075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:34.686215 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:34.686479 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:34.686487 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:34.686632 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 05:19:34.686745 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 05:19:34.686916 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260812 05:19:34.687641 6089 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39357
I20260812 05:19:34.687762 6089 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:34.687975 6089 heartbeater.cc:511] Master 127.18.166.254:39357 requested a full tablet report, sending...
I20260812 05:19:34.688520 5885 ts_manager.cc:194] Registered new tserver with Master: b6bb4adefbd2465a9d0a3b109b407e47 (127.18.166.193:37969)
I20260812 05:19:34.689157 5885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:39401
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:19:34.708554 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39357
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:34.790316 6095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:34.790568 6095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:34.790612 6095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:34.791875 6095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:34.791939 6095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:34.791985 6095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:34.793711 6095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:34.793980 6095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:34.794229 6095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:34.799419 6104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.799423 6101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.799437 6102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:34.799800 6095 server_base.cc:1034] running on GCE node
I20260812 05:19:34.800032 6095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:34.800290 6095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:34.801468 6095 hybrid_clock.cc:648] HybridClock initialized: now 1786511974801453 us; error 26 us; skew 500 ppm
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:34.804118 6095 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:19:34.804522 6095 webserver.cc:460] Webserver started at http://127.18.166.194:43887/ using document root <none> and password file <none>
I20260812 05:19:34.804726 6095 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:34.804767 6095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:34.804858 6095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:34.805936 6095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "cfe9929c46db4b0596b71c10f54da7bb"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "76d1e3f9cda683a663b194adcd41e75f"
server_key_iv: "8ec1ab9409125204866af27da3b976df"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.806232 6095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "cfe9929c46db4b0596b71c10f54da7bb"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "76d1e3f9cda683a663b194adcd41e75f"
server_key_iv: "8ec1ab9409125204866af27da3b976df"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.807507 6095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:19:34.808180 6112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.808365 6095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:34.808452 6095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "cfe9929c46db4b0596b71c10f54da7bb"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "76d1e3f9cda683a663b194adcd41e75f"
server_key_iv: "8ec1ab9409125204866af27da3b976df"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.808508 6095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:34.828284 6095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:34.830307 6095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:34.830425 6095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:34.830615 6095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:34.830942 6095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:34.830978 6095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.831000 6095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:34.831046 6095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.837001 6095 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:38421
I20260812 05:19:34.837087 6231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:38421 every 8 connection(s)
I20260812 05:19:34.837437 6095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:34.843776 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6095
I20260812 05:19:34.843919 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:19:34.844168 19099 external_mini_cluster.cc:1428] Setting key 5cfbc9d3e78ca98c499bbe87e76bcd75
I20260812 05:19:34.847317 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:34.838237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:46419 (local address 127.18.166.254:39357)
0812 05:19:34.838302 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:34.838303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:34.838506 (+ 203us) server_negotiation.cc:407] Connection header received
0812 05:19:34.838977 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:34.838978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:34.838989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:34.839041 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:34.839946 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.840438 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.841377 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.841574 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.841635 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:34.843788 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:34.843790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:34.843790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:34.843798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:34.845614 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:34.846136 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:34.846138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:34.846138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:34.846200 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:34.846542 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:34.846544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:34.846545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:34.846742 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:34.846827 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:34.847094 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 05:19:34.847197 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:19:34.847337 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:34.838230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39357 (local address 127.18.166.194:46419)
0812 05:19:34.838381 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:34.838417 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:34.838896 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:34.839093 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:34.839096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:34.839675 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:34.839870 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:34.839875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.840517 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.840517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:34.841304 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:34.841306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:34.841661 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:34.841662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:34.841770 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:34.842233 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:34.842241 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:34.843665 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:34.845694 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:34.845703 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:34.845705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:34.846035 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:34.846279 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:34.846280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:34.846280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:34.846432 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:34.846822 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:34.846827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:34.846950 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 05:19:34.847038 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:19:34.847173 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260812 05:19:34.847965 6232 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39357
I20260812 05:19:34.848071 6232 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:34.848272 6232 heartbeater.cc:511] Master 127.18.166.254:39357 requested a full tablet report, sending...
I20260812 05:19:34.848668 5885 ts_manager.cc:194] Registered new tserver with Master: cfe9929c46db4b0596b71c10f54da7bb (127.18.166.194:38421)
I20260812 05:19:34.849226 5885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:46419
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:19:34.870560 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39357
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:34.952008 6237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:34.952239 6237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:34.952266 6237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:34.953527 6237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:34.953572 6237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:34.953629 6237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:34.955132 6237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:34.955395 6237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:34.955642 6237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:34.960726 6246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.960758 6244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:34.960722 6243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:34.960917 6237 server_base.cc:1034] running on GCE node
I20260812 05:19:34.961230 6237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:34.961539 6237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:34.962723 6237 hybrid_clock.cc:648] HybridClock initialized: now 1786511974962716 us; error 32 us; skew 500 ppm
Aug 12 05:19:34 dist-test-slave-srgv krb5kdc[5839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:34.968919 6237 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:19:34.969400 6237 webserver.cc:460] Webserver started at http://127.18.166.195:41755/ using document root <none> and password file <none>
I20260812 05:19:34.969698 6237 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:34.969746 6237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:34.969893 6237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:34.970978 6237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "5fc3872c36f940d1b4a86e00dd1c7e68"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "7fe84d2d9f166d42eda222a874dfb8bd"
server_key_iv: "4d3d5516a2017f3c959af2f25179c69b"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.971356 6237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "5fc3872c36f940d1b4a86e00dd1c7e68"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "7fe84d2d9f166d42eda222a874dfb8bd"
server_key_iv: "4d3d5516a2017f3c959af2f25179c69b"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.972733 6237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:34.973524 6253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.973699 6237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:34.973801 6237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "5fc3872c36f940d1b4a86e00dd1c7e68"
format_stamp: "Formatted at 2026-08-12 05:19:34 on dist-test-slave-srgv"
server_key: "7fe84d2d9f166d42eda222a874dfb8bd"
server_key_iv: "4d3d5516a2017f3c959af2f25179c69b"
server_key_version: "encryptionkey@0"
I20260812 05:19:34.973886 6237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:34.990772 6237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:34.993022 6237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:34.993197 6237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:34.993551 6237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:34.993911 6237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:34.993948 6237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:34.993970 6237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:34.993983 6237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.000314 6237 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:46361
I20260812 05:19:35.000360 6371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:46361 every 8 connection(s)
I20260812 05:19:35.000691 6237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:19:35.005851 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6237
I20260812 05:19:35.005970 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:19:35.006167 19099 external_mini_cluster.cc:1428] Setting key 55c26707b53c4768c78808825ef59297
Aug 12 05:19:35 dist-test-slave-srgv krb5kdc[5839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:35.010789 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.001435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:41175 (local address 127.18.166.254:39357)
0812 05:19:35.001492 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.001494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.001702 (+ 208us) server_negotiation.cc:407] Connection header received
0812 05:19:35.002223 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.002224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.002235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.002277 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.003257 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.003759 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.004837 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.005072 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.005167 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.007836 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.007838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.007838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:35.007845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.009480 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:35.009883 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:35.009884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:35.009884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:35.009940 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:35.010197 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:35.010198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:35.010199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:35.010328 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.010398 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.010648 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.010723 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:19:35.010978 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.001394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39357 (local address 127.18.166.195:41175)
0812 05:19:35.001591 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.001629 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.002107 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.002333 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.002336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.002932 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:35.003126 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.003131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.003856 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.003857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.004711 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.004714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.005195 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.005197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.005328 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.005929 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:35.005942 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:35.007752 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:35.009569 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:35.009576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:35.009578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:35.009816 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:35.009998 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:35.009998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:35.009999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:35.010099 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:35.010399 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:35.010404 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:35.010528 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.010617 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.010754 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260812 05:19:35.011520 6372 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39357
I20260812 05:19:35.011626 6372 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:35.011828 6372 heartbeater.cc:511] Master 127.18.166.254:39357 requested a full tablet report, sending...
I20260812 05:19:35.012449 5885 ts_manager.cc:194] Registered new tserver with Master: 5fc3872c36f940d1b4a86e00dd1c7e68 (127.18.166.195:46361)
I20260812 05:19:35.012849 5885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:41175
I20260812 05:19:35.018075 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:19:35.025782 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.019430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.18.166.254:39357)
0812 05:19:35.019500 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.019502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.019611 (+ 109us) server_negotiation.cc:407] Connection header received
0812 05:19:35.019654 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.019655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.019665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.019725 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.020229 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.020702 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.021384 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.021632 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.021690 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.022986 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.022987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.022988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:35.022993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.024524 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:35.024903 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:35.024904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:35.024904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:35.024958 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:35.025139 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:35.025140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:35.025140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:35.025291 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.025373 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.025662 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.025716 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:19:35.027472 5885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:19:35.029066 5885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:19:35.036150 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.031770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.18.166.194:38421)
0812 05:19:35.032492 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.032497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.032506 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:19:35.032546 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.032547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.032563 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.032672 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.032809 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.033876 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035026 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.035715 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035791 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.035837 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.036013 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.036056 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":102,"threads_started":1}
I20260812 05:19:35.036173 6385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.031607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38421 (local address 127.0.0.1:41974)
0812 05:19:35.031789 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.031834 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.031903 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.032692 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.032693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.032694 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.032744 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.032745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.033973 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.033974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.034930 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.034934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.035797 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.035891 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.035907 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.035982 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.036052 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":54,"threads_started":1}
I20260812 05:19:35.036180 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.031619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:37969 (local address 127.0.0.1:58248)
0812 05:19:35.031975 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.031987 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.032044 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.032372 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.032376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.032378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.032454 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.032456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.033834 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.033835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.034883 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.034884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.035923 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.035986 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.035989 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.036035 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.036075 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260812 05:19:35.036377 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.032020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.18.166.195:46361)
0812 05:19:35.032220 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.032229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.032238 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:19:35.032290 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.032292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.032307 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.032389 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.032615 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.033388 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.034917 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.035672 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035751 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.035817 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.036225 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.036280 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":133,"threads_started":1}
I20260812 05:19:35.036384 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.031830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.18.166.193:37969)
0812 05:19:35.032183 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.032187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.032195 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:19:35.032229 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.032231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.032243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.032347 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.032544 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.033727 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.034962 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.035736 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035829 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.035889 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.036222 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.036279 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":70,"threads_started":1}
I20260812 05:19:35.036182 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.031796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:46361 (local address 127.0.0.1:44418)
0812 05:19:35.031976 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.031987 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.032044 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.032475 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.032476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.032479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.032530 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.032532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.033515 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.033516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.034819 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.034822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.035856 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.035857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.035926 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.035931 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.035999 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.036055 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":4,"thread_start_us":87,"threads_started":1}
I20260812 05:19:35.037175 6021 tablet_service.cc:1467] Processing CreateTablet for tablet b872a322df574b108888d8e15c600c44 (DEFAULT_TABLE table=test-table [id=d02f60b3b4954ac1bd538857b051852e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:35.037173 6304 tablet_service.cc:1467] Processing CreateTablet for tablet b872a322df574b108888d8e15c600c44 (DEFAULT_TABLE table=test-table [id=d02f60b3b4954ac1bd538857b051852e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:35.037586 6304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b872a322df574b108888d8e15c600c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.037586 6021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b872a322df574b108888d8e15c600c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.037730 6164 tablet_service.cc:1467] Processing CreateTablet for tablet b872a322df574b108888d8e15c600c44 (DEFAULT_TABLE table=test-table [id=d02f60b3b4954ac1bd538857b051852e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:35.038053 6164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b872a322df574b108888d8e15c600c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.040679 6390 tablet_bootstrap.cc:492] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Bootstrap starting.
I20260812 05:19:35.041116 6391 tablet_bootstrap.cc:492] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47: Bootstrap starting.
I20260812 05:19:35.041495 6390 tablet_bootstrap.cc:654] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.041640 6392 tablet_bootstrap.cc:492] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Bootstrap starting.
I20260812 05:19:35.041811 6391 tablet_bootstrap.cc:654] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.042168 6390 log.cc:824] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.042300 6392 tablet_bootstrap.cc:654] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.042376 6391 log.cc:824] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.042917 6392 log.cc:824] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.043116 6390 tablet_bootstrap.cc:492] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: No bootstrap required, opened a new log
I20260812 05:19:35.043193 6390 ts_tablet_manager.cc:1397] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:19:35.043231 6391 tablet_bootstrap.cc:492] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47: No bootstrap required, opened a new log
I20260812 05:19:35.043306 6391 ts_tablet_manager.cc:1397] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:19:35.043819 6392 tablet_bootstrap.cc:492] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: No bootstrap required, opened a new log
I20260812 05:19:35.043889 6392 ts_tablet_manager.cc:1397] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:35.044740 6390 raft_consensus.cc:348] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.044909 6390 raft_consensus.cc:374] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.044955 6390 raft_consensus.cc:729] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe9929c46db4b0596b71c10f54da7bb, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.045063 6390 consensus_queue.cc:260] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.045383 6390 ts_tablet_manager.cc:1428] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:19:35.045368 6391 raft_consensus.cc:348] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.045562 6232 heartbeater.cc:503] Master 127.18.166.254:39357 was elected leader, sending a full tablet report...
I20260812 05:19:35.045598 6391 raft_consensus.cc:374] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.045652 6391 raft_consensus.cc:729] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6bb4adefbd2465a9d0a3b109b407e47, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.045764 6391 consensus_queue.cc:260] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.045915 6392 raft_consensus.cc:348] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.046068 6392 raft_consensus.cc:374] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.046056 6089 heartbeater.cc:503] Master 127.18.166.254:39357 was elected leader, sending a full tablet report...
I20260812 05:19:35.046070 6391 ts_tablet_manager.cc:1428] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 05:19:35.046108 6392 raft_consensus.cc:729] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc3872c36f940d1b4a86e00dd1c7e68, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.046236 6392 consensus_queue.cc:260] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.046499 6392 ts_tablet_manager.cc:1428] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:19:35.046499 6372 heartbeater.cc:503] Master 127.18.166.254:39357 was elected leader, sending a full tablet report...
W20260812 05:19:35.088049 6233 tablet.cc:2369] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:19:35.106834 6398 raft_consensus.cc:482] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:35.106962 6398 raft_consensus.cc:504] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.107311 6398 leader_election.cc:288] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfe9929c46db4b0596b71c10f54da7bb (127.18.166.194:38421), b6bb4adefbd2465a9d0a3b109b407e47 (127.18.166.193:37969)
I20260812 05:19:35.110206 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.107503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:37969 (local address 127.18.166.195:58729)
0812 05:19:35.107579 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.107592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.107657 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.107810 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.107811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.107813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.107890 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.107892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.108545 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.108546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.109300 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.109302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.109926 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.109927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.110014 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.110017 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.110086 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.110139 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 05:19:35.110206 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.107613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:58729 (local address 127.18.166.193:37969)
0812 05:19:35.107673 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.107675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.107687 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:19:35.107713 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.107714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.107722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.107802 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.107968 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.108455 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.109401 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.109834 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.109926 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.109935 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.110091 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.110131 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:19:35.110432 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.107567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:41933 (local address 127.18.166.194:38421)
0812 05:19:35.107634 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.107636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.107855 (+ 219us) server_negotiation.cc:407] Connection header received
0812 05:19:35.107908 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.107909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.107921 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.107996 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.108197 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.108659 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.109653 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.110030 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.110113 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.110120 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.110302 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.110353 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:19:35.110579 6041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b872a322df574b108888d8e15c600c44" candidate_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" is_pre_election: true
I20260812 05:19:35.110649 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.107510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:38421 (local address 127.18.166.195:41933)
0812 05:19:35.107773 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.107781 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.107855 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.108006 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.108006 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.108008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.108098 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.108099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.108747 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.108748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.109553 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.109555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.110112 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.110113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.110186 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.110190 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.110519 (+ 329us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.110560 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":84,"thread_start_us":133,"threads_started":1}
I20260812 05:19:35.110759 6041 raft_consensus.cc:2454] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc3872c36f940d1b4a86e00dd1c7e68 in term 0.
I20260812 05:19:35.110952 6184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b872a322df574b108888d8e15c600c44" candidate_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe9929c46db4b0596b71c10f54da7bb" is_pre_election: true
I20260812 05:19:35.110991 6256 leader_election.cc:302] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc3872c36f940d1b4a86e00dd1c7e68, b6bb4adefbd2465a9d0a3b109b407e47; no voters:
I20260812 05:19:35.111100 6184 raft_consensus.cc:2454] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc3872c36f940d1b4a86e00dd1c7e68 in term 0.
I20260812 05:19:35.111112 6398 raft_consensus.cc:2781] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:19:35.111166 6398 raft_consensus.cc:482] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:19:35.111199 6398 raft_consensus.cc:3037] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:35.111989 6398 raft_consensus.cc:504] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.112136 6398 leader_election.cc:288] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [CANDIDATE]: Term 1 election: Requested vote from peers cfe9929c46db4b0596b71c10f54da7bb (127.18.166.194:38421), b6bb4adefbd2465a9d0a3b109b407e47 (127.18.166.193:37969)
I20260812 05:19:35.112437 6184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b872a322df574b108888d8e15c600c44" candidate_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe9929c46db4b0596b71c10f54da7bb"
I20260812 05:19:35.112529 6184 raft_consensus.cc:3037] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:35.113251 6184 raft_consensus.cc:2454] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc3872c36f940d1b4a86e00dd1c7e68 in term 1.
I20260812 05:19:35.113457 6254 leader_election.cc:302] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc3872c36f940d1b4a86e00dd1c7e68, cfe9929c46db4b0596b71c10f54da7bb; no voters:
I20260812 05:19:35.113400 6041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b872a322df574b108888d8e15c600c44" candidate_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6bb4adefbd2465a9d0a3b109b407e47"
I20260812 05:19:35.113518 6041 raft_consensus.cc:3037] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:35.113549 6398 raft_consensus.cc:2781] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:35.113771 6398 raft_consensus.cc:686] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc3872c36f940d1b4a86e00dd1c7e68, State: Running, Role: LEADER
I20260812 05:19:35.113874 6398 consensus_queue.cc:237] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } }
I20260812 05:19:35.114419 6041 raft_consensus.cc:2454] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc3872c36f940d1b4a86e00dd1c7e68 in term 1.
I20260812 05:19:35.114741 5883 catalog_manager.cc:5495] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc3872c36f940d1b4a86e00dd1c7e68 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc3872c36f940d1b4a86e00dd1c7e68" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 46361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 } health_report { overall_health: UNKNOWN } } }
I20260812 05:19:35.128219 6389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.126029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.18.166.195:46361)
0812 05:19:35.126089 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.126091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.126101 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:35.126154 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.126155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.126171 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.126232 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:19:35.126312 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.126766 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.127496 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.127690 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.127745 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.127874 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:19:35.127954 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:19:35.128022 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.128067 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.128078 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:19:35.130127 6041 raft_consensus.cc:1261] T b872a322df574b108888d8e15c600c44 P b6bb4adefbd2465a9d0a3b109b407e47 [term 1 FOLLOWER]: Refusing update from remote peer 5fc3872c36f940d1b4a86e00dd1c7e68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:35.130127 6184 raft_consensus.cc:1261] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 1 FOLLOWER]: Refusing update from remote peer 5fc3872c36f940d1b4a86e00dd1c7e68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:35.130451 6400 consensus_queue.cc:1035] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6bb4adefbd2465a9d0a3b109b407e47" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 37969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:35.130584 6398 consensus_queue.cc:1035] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfe9929c46db4b0596b71c10f54da7bb" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 38421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:35.133492 6403 mvcc.cc:205] Tried to move back new op lower bound from 7317553050130276352 to 7317553050067095552. Current Snapshot: MvccSnapshot[applied={T|T < 7317553050130276352}]
I20260812 05:19:35.133759 6405 mvcc.cc:205] Tried to move back new op lower bound from 7317553050130276352 to 7317553050067095552. Current Snapshot: MvccSnapshot[applied={T|T < 7317553050130276352}]
I20260812 05:19:35.136363 5885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:35.136468 5885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:35.137269 5885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79c315674600471486b053dc5d1ae460: Sending DeleteTablet for 3 replicas of tablet b872a322df574b108888d8e15c600c44
I20260812 05:19:35.137633 6021 tablet_service.cc:1514] Processing DeleteTablet for tablet b872a322df574b108888d8e15c600c44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:35 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:58248
I20260812 05:19:35.137708 6164 tablet_service.cc:1514] Processing DeleteTablet for tablet b872a322df574b108888d8e15c600c44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:35 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:41974
I20260812 05:19:35.137871 6304 tablet_service.cc:1514] Processing DeleteTablet for tablet b872a322df574b108888d8e15c600c44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:35 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:44418
I20260812 05:19:35.138020 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5953
I20260812 05:19:35.143064 6414 tablet_replica.cc:330] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: stopping tablet replica
I20260812 05:19:35.143231 6414 raft_consensus.cc:2229] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:35.143374 6414 raft_consensus.cc:2258] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:35.143594 6415 tablet_replica.cc:330] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: stopping tablet replica
I20260812 05:19:35.143700 6415 raft_consensus.cc:2229] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:19:35.143836 6415 raft_consensus.cc:2258] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:35.144250 6415 ts_tablet_manager.cc:1905] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:35.145784 6414 ts_tablet_manager.cc:1905] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:35.146032 6415 ts_tablet_manager.cc:1918] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:35.146178 6415 log.cc:1196] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/b872a322df574b108888d8e15c600c44
I20260812 05:19:35.146533 6415 ts_tablet_manager.cc:1939] T b872a322df574b108888d8e15c600c44 P 5fc3872c36f940d1b4a86e00dd1c7e68: Deleting consensus metadata
I20260812 05:19:35.147099 5869 catalog_manager.cc:4841] TS 5fc3872c36f940d1b4a86e00dd1c7e68 (127.18.166.195:46361): tablet b872a322df574b108888d8e15c600c44 (table test-table [id=d02f60b3b4954ac1bd538857b051852e]) successfully deleted
I20260812 05:19:35.147619 6414 ts_tablet_manager.cc:1918] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:35.147706 6414 log.cc:1196] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/b872a322df574b108888d8e15c600c44
W20260812 05:19:35.147956 5871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:37969 (error 108)
I20260812 05:19:35.148005 6414 ts_tablet_manager.cc:1939] T b872a322df574b108888d8e15c600c44 P cfe9929c46db4b0596b71c10f54da7bb: Deleting consensus metadata
I20260812 05:19:35.148450 5869 catalog_manager.cc:4841] TS cfe9929c46db4b0596b71c10f54da7bb (127.18.166.194:38421): tablet b872a322df574b108888d8e15c600c44 (table test-table [id=d02f60b3b4954ac1bd538857b051852e]) successfully deleted
I20260812 05:19:35.148460 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.148336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:37969 (local address 127.0.0.1:58264)
0812 05:19:35.148389 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.148423 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:37969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:19:35.148720 5871 catalog_manager.cc:4584] TS b6bb4adefbd2465a9d0a3b109b407e47 (127.18.166.193:37969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b872a322df574b108888d8e15c600c44: Network error: Client connection negotiation failed: client connection to 127.18.166.193:37969: connect: Connection refused (error 111)
I20260812 05:19:35.148833 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6095
I20260812 05:19:35.155243 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6237
I20260812 05:19:35.161528 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 5852
2026-08-12T05:19:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2014 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T05:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:35Z Disabled control of system clock
I20260812 05:19:35.190943 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42115
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:42115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:35.273113 6420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:35.273353 6420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:35.273375 6420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:35.274575 6420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:35.274621 6420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:35.274633 6420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:35.274650 6420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:35.274662 6420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:35.276121 6420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:42115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:42115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:35.276394 6420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:35.276702 6420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:35.281661 6428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.281669 6426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.281705 6425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:35.282070 6420 server_base.cc:1034] running on GCE node
I20260812 05:19:35.282297 6420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:35.282588 6420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:35.283741 6420 hybrid_clock.cc:648] HybridClock initialized: now 1786511975283706 us; error 51 us; skew 500 ppm
I20260812 05:19:35.285107 6420 webserver.cc:460] Webserver started at http://127.18.166.254:34607/ using document root <none> and password file <none>
I20260812 05:19:35.285316 6420 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:35.285358 6420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:35.285517 6420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:35.286574 6420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "6f47c75c357440cf8061ae317ee6e46c"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "9d70618b5bc55985fe85b55b0c0d0680"
server_key_iv: "86ffe40f887aa5460fbe252e7b4c4ece"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.286878 6420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "6f47c75c357440cf8061ae317ee6e46c"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "9d70618b5bc55985fe85b55b0c0d0680"
server_key_iv: "86ffe40f887aa5460fbe252e7b4c4ece"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.288146 6420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:19:35.288815 6434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.288985 6420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:35.289095 6420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "6f47c75c357440cf8061ae317ee6e46c"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "9d70618b5bc55985fe85b55b0c0d0680"
server_key_iv: "86ffe40f887aa5460fbe252e7b4c4ece"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.289172 6420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:35.317112 6420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:35.317523 6420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:35.317693 6420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:35.321604 6420 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:42115
I20260812 05:19:35.321662 6491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:42115 every 8 connection(s)
I20260812 05:19:35.322021 6420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:35.322983 6492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.325323 6492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c: Bootstrap starting.
I20260812 05:19:35.325991 6492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.325997 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6420
I20260812 05:19:35.326116 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:35.326359 19099 external_mini_cluster.cc:1428] Setting key b75a4ba171ef73afd4af9f7126272caa
I20260812 05:19:35.326380 6492 log.cc:824] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.327221 6492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c: No bootstrap required, opened a new log
I20260812 05:19:35.329169 6492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47c75c357440cf8061ae317ee6e46c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42115 } }
I20260812 05:19:35.329325 6492 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.329371 6492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47c75c357440cf8061ae317ee6e46c, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.329546 6492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47c75c357440cf8061ae317ee6e46c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42115 } }
I20260812 05:19:35.329631 6492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:35.329667 6492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:35.329720 6492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:35.330274 6492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47c75c357440cf8061ae317ee6e46c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42115 } }
I20260812 05:19:35.330389 6492 leader_election.cc:302] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f47c75c357440cf8061ae317ee6e46c; no voters:
I20260812 05:19:35.330591 6492 leader_election.cc:288] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:35.330878 6492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:35.330624 6499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:35.331158 6499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [term 1 LEADER]: Becoming Leader. State: Replica: 6f47c75c357440cf8061ae317ee6e46c, State: Running, Role: LEADER
I20260812 05:19:35.331249 6499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47c75c357440cf8061ae317ee6e46c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42115 } }
I20260812 05:19:35.331672 6500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f47c75c357440cf8061ae317ee6e46c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47c75c357440cf8061ae317ee6e46c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42115 } } }
I20260812 05:19:35.331696 6501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f47c75c357440cf8061ae317ee6e46c. Latest consensus state: current_term: 1 leader_uuid: "6f47c75c357440cf8061ae317ee6e46c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47c75c357440cf8061ae317ee6e46c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 42115 } } }
I20260812 05:19:35.331763 6500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:35.331765 6501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:35.332006 6508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:35.332309 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.326877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.18.166.254:42115)
0812 05:19:35.327087 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.327095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.327144 (+ 49us) server_negotiation.cc:407] Connection header received
0812 05:19:35.327594 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.327601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.327665 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.327813 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.328026 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.328904 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.329928 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.330330 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.330464 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.331209 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.331213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.331213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:35.331432 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.331473 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:35.331488 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.331590 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.331959 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.332102 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260812 05:19:35.332465 6508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:35.334308 6508 catalog_manager.cc:1347] Generated new cluster ID: d61a17f935464e999751250dfeb58d22
I20260812 05:19:35.334360 6508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:35.347098 6508 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:35.347652 6508 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:35.363535 6508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c: Generated new TSK 0
I20260812 05:19:35.363770 6508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:35.367724 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42115
--builtin_ntp_servers=127.18.166.212:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:35.452836 6518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:35.453048 6518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:35.453073 6518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:35.454325 6518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:35.454370 6518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:35.454397 6518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:35.455767 6518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:35.456017 6518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:35.456327 6518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:35.461421 6526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.461478 6524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.461429 6523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:35.461658 6518 server_base.cc:1034] running on GCE node
I20260812 05:19:35.461879 6518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:35.462121 6518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:35.463294 6518 hybrid_clock.cc:648] HybridClock initialized: now 1786511975463274 us; error 37 us; skew 500 ppm
I20260812 05:19:35.464564 6518 webserver.cc:460] Webserver started at http://127.18.166.193:44757/ using document root <none> and password file <none>
I20260812 05:19:35.464816 6518 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:35.464886 6518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:35.465039 6518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:35.466229 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "333d1783203e435aa7eca4a4ade9bbcd"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "e49a4a9233bc2c9e3171d3f9fcc21942"
server_key_iv: "7909fc61bb10af8badadf93415ab73ee"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.466598 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "333d1783203e435aa7eca4a4ade9bbcd"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "e49a4a9233bc2c9e3171d3f9fcc21942"
server_key_iv: "7909fc61bb10af8badadf93415ab73ee"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.467944 6518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:19:35.468730 6532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.468910 6518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:35.469012 6518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "333d1783203e435aa7eca4a4ade9bbcd"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "e49a4a9233bc2c9e3171d3f9fcc21942"
server_key_iv: "7909fc61bb10af8badadf93415ab73ee"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.469094 6518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:35.494004 6518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:35.494345 6518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:35.494503 6518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:35.494750 6518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:35.495106 6518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:35.495162 6518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.495210 6518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:35.495268 6518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.501168 6518 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:45407
I20260812 05:19:35.501216 6655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:45407 every 8 connection(s)
I20260812 05:19:35.501611 6518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:19:35.503198 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6518
I20260812 05:19:35.503331 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:19:35.503553 19099 external_mini_cluster.cc:1428] Setting key ceb060b8199606b41b5bf9d3d6e83368
I20260812 05:19:35.504665 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42115
--builtin_ntp_servers=127.18.166.212:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:19:35.507486 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.502344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:55323 (local address 127.18.166.254:42115)
0812 05:19:35.502395 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.502398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.502669 (+ 271us) server_negotiation.cc:407] Connection header received
0812 05:19:35.503204 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.503206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.503221 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.503303 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.504177 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.504898 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.505887 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.506181 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.506239 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.507195 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.507196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.507197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:35.507201 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.507216 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:35.507220 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.507273 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.507376 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.507405 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:35.507735 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.502350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42115 (local address 127.18.166.193:55323)
0812 05:19:35.502547 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.502582 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.503062 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.503402 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.503408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.503419 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:35.504085 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.504091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.504988 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.504989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.505806 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.505808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.506268 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.506269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.506436 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.507031 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:35.507031 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:35.507075 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:35.507098 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:35.507280 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:35.507285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:35.507288 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.507376 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.507480 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260812 05:19:35.508270 6656 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42115
I20260812 05:19:35.508378 6656 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:35.508600 6656 heartbeater.cc:511] Master 127.18.166.254:42115 requested a full tablet report, sending...
I20260812 05:19:35.509205 6454 ts_manager.cc:194] Registered new tserver with Master: 333d1783203e435aa7eca4a4ade9bbcd (127.18.166.193:45407)
I20260812 05:19:35.510226 6454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.193:55323
W20260812 05:19:35.586177 6660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:35.586380 6660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:35.586400 6660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:35.587468 6660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:35.587502 6660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:35.587527 6660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:35.588855 6660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:35.589033 6660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:35.589267 6660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:35.594467 6666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:35.594558 6660 server_base.cc:1034] running on GCE node
W20260812 05:19:35.594475 6668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.594475 6665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:35.594911 6660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:35.595233 6660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:35.596421 6660 hybrid_clock.cc:648] HybridClock initialized: now 1786511975596407 us; error 28 us; skew 500 ppm
I20260812 05:19:35.597872 6660 webserver.cc:460] Webserver started at http://127.18.166.194:41289/ using document root <none> and password file <none>
I20260812 05:19:35.598129 6660 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:35.598206 6660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:35.598363 6660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:35.599452 6660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "682dbbcbde8640ff8ac3f262bfd52a1a"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "531b9f4db07207e5ed3472df3ad30c6f"
server_key_iv: "63f7515f56997b5e1d111d094254f5de"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.599830 6660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "682dbbcbde8640ff8ac3f262bfd52a1a"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "531b9f4db07207e5ed3472df3ad30c6f"
server_key_iv: "63f7515f56997b5e1d111d094254f5de"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.601148 6660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:19:35.601963 6674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.602142 6660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.602248 6660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "682dbbcbde8640ff8ac3f262bfd52a1a"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "531b9f4db07207e5ed3472df3ad30c6f"
server_key_iv: "63f7515f56997b5e1d111d094254f5de"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.602326 6660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:35.620889 6660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:35.621236 6660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:35.621469 6660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:35.621726 6660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:35.622063 6660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:35.622118 6660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.622166 6660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:35.622195 6660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.628082 6660 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:36543
I20260812 05:19:35.628130 6797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:36543 every 8 connection(s)
I20260812 05:19:35.628458 6660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:19:35.630334 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6660
I20260812 05:19:35.630462 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:19:35.630661 19099 external_mini_cluster.cc:1428] Setting key 7931b5679a582dcfc71e58f510f92645
I20260812 05:19:35.631711 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42115
--builtin_ntp_servers=127.18.166.212:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:19:35.634106 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.629228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:43717 (local address 127.18.166.254:42115)
0812 05:19:35.629292 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.629294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.629513 (+ 219us) server_negotiation.cc:407] Connection header received
0812 05:19:35.629999 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.630000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.630011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.630076 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.630809 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.631433 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.632559 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.632879 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.632954 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.633827 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.633828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.633829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:35.633834 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.633850 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:35.633854 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.633904 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.634003 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.634029 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:19:35.634325 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.629205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42115 (local address 127.18.166.194:43717)
0812 05:19:35.629382 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.629443 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.629917 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.630128 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.630132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.630139 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:35.630707 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.630712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.631538 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.631539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.632478 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.632480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.632962 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.632963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.633064 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.633671 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:35.633672 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:35.633699 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:35.633720 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:35.633911 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:35.633916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:35.633919 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.634004 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.634103 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260812 05:19:35.634816 6798 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42115
I20260812 05:19:35.634929 6798 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:35.635108 6798 heartbeater.cc:511] Master 127.18.166.254:42115 requested a full tablet report, sending...
I20260812 05:19:35.635516 6454 ts_manager.cc:194] Registered new tserver with Master: 682dbbcbde8640ff8ac3f262bfd52a1a (127.18.166.194:36543)
I20260812 05:19:35.635895 6454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.194:43717
W20260812 05:19:35.713837 6801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:35.714030 6801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:35.714051 6801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:35.715217 6801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:35.715260 6801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:35.715286 6801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:35.716650 6801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:42115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:35.716899 6801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:35.717197 6801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:35.722007 6806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.722002 6810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:35.722031 6807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:35.722323 6801 server_base.cc:1034] running on GCE node
I20260812 05:19:35.722553 6801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:35.722793 6801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:35.723953 6801 hybrid_clock.cc:648] HybridClock initialized: now 1786511975723935 us; error 30 us; skew 500 ppm
I20260812 05:19:35.725131 6801 webserver.cc:460] Webserver started at http://127.18.166.195:46515/ using document root <none> and password file <none>
I20260812 05:19:35.725378 6801 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:35.725461 6801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:35.725603 6801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:35.726647 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "9a29e16ebdcd41818aa407b2fe2343f8"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "e298694eac77fd1fcab410d3b0b1eee5"
server_key_iv: "8d931d7aea46b81d2720435f1ec84f8f"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.727006 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "9a29e16ebdcd41818aa407b2fe2343f8"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "e298694eac77fd1fcab410d3b0b1eee5"
server_key_iv: "8d931d7aea46b81d2720435f1ec84f8f"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.728291 6801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:19:35.729008 6816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.729172 6801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:35.729296 6801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "9a29e16ebdcd41818aa407b2fe2343f8"
format_stamp: "Formatted at 2026-08-12 05:19:35 on dist-test-slave-srgv"
server_key: "e298694eac77fd1fcab410d3b0b1eee5"
server_key_iv: "8d931d7aea46b81d2720435f1ec84f8f"
server_key_version: "encryptionkey@0"
I20260812 05:19:35.729382 6801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:35.774331 6801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:35.774677 6801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:35.774833 6801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:35.775086 6801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:35.775444 6801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:35.775497 6801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.775532 6801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:35.775563 6801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:35.781337 6801 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:35855
I20260812 05:19:35.781456 6935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:35855 every 8 connection(s)
I20260812 05:19:35.781790 6801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:19:35.787328 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.782586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:44363 (local address 127.18.166.254:42115)
0812 05:19:35.782645 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.782647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.782943 (+ 296us) server_negotiation.cc:407] Connection header received
0812 05:19:35.783507 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.783508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.783520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.783574 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.784347 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.784857 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.785866 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.786070 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.786150 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.786964 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.786965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.786966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:35.786970 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.786985 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:35.786989 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.787042 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.787215 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.787244 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:19:35.787551 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.782572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:42115 (local address 127.18.166.195:44363)
0812 05:19:35.782816 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.782860 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.783398 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.783654 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.783658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.783668 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:35.784251 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.784257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.784961 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.784962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.785770 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.785772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.786161 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.786162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.786301 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.786822 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:35.786823 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:35.786850 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:35.786872 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:35.787048 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:35.787053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:35.787056 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.787212 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.787315 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260812 05:19:35.787833 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6801
I20260812 05:19:35.787951 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:19:35.788112 6936 heartbeater.cc:347] Connected to a master server at 127.18.166.254:42115
I20260812 05:19:35.788183 19099 external_mini_cluster.cc:1428] Setting key c8b24364865dd735e09e3af99a9bc4cf
I20260812 05:19:35.788228 6936 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:35.788439 6936 heartbeater.cc:511] Master 127.18.166.254:42115 requested a full tablet report, sending...
I20260812 05:19:35.788861 6454 ts_manager.cc:194] Registered new tserver with Master: 9a29e16ebdcd41818aa407b2fe2343f8 (127.18.166.195:35855)
I20260812 05:19:35.789181 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:19:35.789355 6454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.195:44363
I20260812 05:19:35.793653 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.790654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.18.166.254:42115)
0812 05:19:35.790690 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.790692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.790828 (+ 136us) server_negotiation.cc:407] Connection header received
0812 05:19:35.790901 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.790902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.790912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.790967 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:35.791160 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.791642 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.792414 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.792618 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.792696 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.793361 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:35.793363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:35.793363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:35.793367 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:35.793380 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:35.793384 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:35.793493 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.793554 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.793564 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 05:19:35.794914 6454 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:19:35.796438 6454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:19:35.802876 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.798550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:35855 (local address 127.0.0.1:46266)
0812 05:19:35.798675 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.798718 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.798788 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.799094 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.799097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.799099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.799189 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.799191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.800508 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.800509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.801690 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.801693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.802544 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.802622 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.802636 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.802730 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.802785 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":33,"thread_start_us":87,"threads_started":1}
I20260812 05:19:35.803022 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.798816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.18.166.194:36543)
0812 05:19:35.799012 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.799017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.799062 (+ 45us) server_negotiation.cc:407] Connection header received
0812 05:19:35.799145 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.799147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.799161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.799328 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.799523 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.800294 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802120 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.802676 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802749 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.802828 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.802904 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.802945 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
I20260812 05:19:35.803025 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.798598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:36543 (local address 127.0.0.1:43278)
0812 05:19:35.799006 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.799017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.799074 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.799395 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.799395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.799397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.799441 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.799443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.800383 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.800384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.802034 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.802036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.802758 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.802821 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.802825 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.802892 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.802935 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":27,"thread_start_us":78,"threads_started":1}
I20260812 05:19:35.802892 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.798653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.18.166.195:35855)
0812 05:19:35.798858 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.798863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.798875 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:19:35.798916 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.798918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.798933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.799015 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.799269 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.800425 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.801757 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.802451 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802533 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.802629 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.802759 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.802795 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
I20260812 05:19:35.803423 6947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.798779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:45407 (local address 127.0.0.1:60694)
0812 05:19:35.799316 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:35.799325 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:35.799362 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:35.799627 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:35.799628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:35.799630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:35.799670 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.799671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.800831 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.800832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.801627 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:35.801630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.802980 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:35.803043 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.803046 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:35.803317 (+ 271us) client_negotiation.cc:240] Negotiation successful
0812 05:19:35.803359 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":64,"threads_started":1}
I20260812 05:19:35.803900 6949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:35.798982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.18.166.193:45407)
0812 05:19:35.799479 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:35.799483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:35.799491 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:19:35.799528 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:35.799530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:35.799544 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:35.799614 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:35.799744 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.800744 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.801723 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:35.802451 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:35.802533 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:35.802619 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:35.803528 (+ 909us) server_negotiation.cc:299] Negotiation successful
0812 05:19:35.803574 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":88,"threads_started":1}
I20260812 05:19:35.803777 6725 tablet_service.cc:1467] Processing CreateTablet for tablet 2979fc1d9371411db2ede0d4e0a743d4 (DEFAULT_TABLE table=test-table [id=773881d6fc874ce0bde9c70402a5f4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:35.804008 6865 tablet_service.cc:1467] Processing CreateTablet for tablet 2979fc1d9371411db2ede0d4e0a743d4 (DEFAULT_TABLE table=test-table [id=773881d6fc874ce0bde9c70402a5f4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:35.804220 6725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2979fc1d9371411db2ede0d4e0a743d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.804378 6865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2979fc1d9371411db2ede0d4e0a743d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.804561 6583 tablet_service.cc:1467] Processing CreateTablet for tablet 2979fc1d9371411db2ede0d4e0a743d4 (DEFAULT_TABLE table=test-table [id=773881d6fc874ce0bde9c70402a5f4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:35.804890 6583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2979fc1d9371411db2ede0d4e0a743d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:35.807243 6950 tablet_bootstrap.cc:492] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Bootstrap starting.
I20260812 05:19:35.807694 6951 tablet_bootstrap.cc:492] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Bootstrap starting.
I20260812 05:19:35.807891 6950 tablet_bootstrap.cc:654] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.808202 6952 tablet_bootstrap.cc:492] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: Bootstrap starting.
I20260812 05:19:35.808349 6951 tablet_bootstrap.cc:654] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.808440 6950 log.cc:824] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.808671 6951 log.cc:824] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.808842 6952 tablet_bootstrap.cc:654] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:35.809181 6952 log.cc:824] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:35.809376 6950 tablet_bootstrap.cc:492] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: No bootstrap required, opened a new log
I20260812 05:19:35.809501 6950 ts_tablet_manager.cc:1397] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:19:35.809605 6951 tablet_bootstrap.cc:492] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: No bootstrap required, opened a new log
I20260812 05:19:35.809710 6951 ts_tablet_manager.cc:1397] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:35.810004 6952 tablet_bootstrap.cc:492] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: No bootstrap required, opened a new log
I20260812 05:19:35.810186 6952 ts_tablet_manager.cc:1397] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:35.811676 6951 raft_consensus.cc:348] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:35.811834 6951 raft_consensus.cc:374] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.811878 6951 raft_consensus.cc:729] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 682dbbcbde8640ff8ac3f262bfd52a1a, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.812007 6951 consensus_queue.cc:260] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:35.812281 6951 ts_tablet_manager.cc:1428] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:19:35.812250 6950 raft_consensus.cc:348] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:35.812407 6950 raft_consensus.cc:374] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.812436 6798 heartbeater.cc:503] Master 127.18.166.254:42115 was elected leader, sending a full tablet report...
I20260812 05:19:35.812444 6950 raft_consensus.cc:729] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a29e16ebdcd41818aa407b2fe2343f8, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.812554 6950 consensus_queue.cc:260] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:35.812789 6950 ts_tablet_manager.cc:1428] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:19:35.812911 6936 heartbeater.cc:503] Master 127.18.166.254:42115 was elected leader, sending a full tablet report...
I20260812 05:19:35.812847 6952 raft_consensus.cc:348] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:35.813246 6952 raft_consensus.cc:374] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:35.813288 6952 raft_consensus.cc:729] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333d1783203e435aa7eca4a4ade9bbcd, State: Initialized, Role: FOLLOWER
I20260812 05:19:35.813391 6952 consensus_queue.cc:260] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:35.813668 6952 ts_tablet_manager.cc:1428] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 05:19:35.813799 6656 heartbeater.cc:503] Master 127.18.166.254:42115 was elected leader, sending a full tablet report...
W20260812 05:19:35.878993 6799 tablet.cc:2369] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:19:36.002377 6657 tablet.cc:2369] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:19:36.032488 6937 tablet.cc:2369] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:19:36.125792 6957 raft_consensus.cc:482] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:36.125955 6957 raft_consensus.cc:504] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:36.126355 6957 leader_election.cc:288] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 333d1783203e435aa7eca4a4ade9bbcd (127.18.166.193:45407), 682dbbcbde8640ff8ac3f262bfd52a1a (127.18.166.194:36543)
I20260812 05:19:36.130546 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:36.126683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:45407 (local address 127.18.166.195:51749)
0812 05:19:36.126928 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:36.126936 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:36.127008 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:36.127177 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:36.127178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:36.127180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:36.127244 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:36.127245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.128095 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.128097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:36.129482 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:36.129483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.130250 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.130251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:36.130334 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:36.130337 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:36.130433 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 05:19:36.130460 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":105,"thread_start_us":111,"threads_started":1}
I20260812 05:19:36.130573 6949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:36.126789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:51749 (local address 127.18.166.193:45407)
0812 05:19:36.126837 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:36.126839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:36.127036 (+ 197us) server_negotiation.cc:407] Connection header received
0812 05:19:36.127069 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:36.127070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:36.127092 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:36.127174 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:36.127328 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.128007 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.129593 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.130151 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.130238 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:36.130247 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:36.130446 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 05:19:36.130483 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:19:36.130717 6948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:36.126814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:33441 (local address 127.18.166.194:36543)
0812 05:19:36.126866 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:36.126868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:36.126879 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:19:36.126918 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:36.126918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:36.126929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:36.126981 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:36.127174 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.127966 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.129562 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.130152 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.130238 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:36.130247 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:36.130582 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 05:19:36.130632 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:19:36.130545 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:36.126681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:36543 (local address 127.18.166.195:33441)
0812 05:19:36.126755 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:36.126770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:36.126840 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:36.127010 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:36.127011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:36.127012 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:36.127086 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:36.127088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.128095 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.128097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:36.129477 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:36.129481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.130249 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.130250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:36.130334 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:36.130337 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:36.130393 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:19:36.130447 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":68}
I20260812 05:19:36.131006 6606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2979fc1d9371411db2ede0d4e0a743d4" candidate_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333d1783203e435aa7eca4a4ade9bbcd" is_pre_election: true
I20260812 05:19:36.131173 6606 raft_consensus.cc:2454] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a29e16ebdcd41818aa407b2fe2343f8 in term 0.
I20260812 05:19:36.131157 6747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2979fc1d9371411db2ede0d4e0a743d4" candidate_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" is_pre_election: true
I20260812 05:19:36.131358 6747 raft_consensus.cc:2454] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a29e16ebdcd41818aa407b2fe2343f8 in term 0.
I20260812 05:19:36.131417 6818 leader_election.cc:302] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333d1783203e435aa7eca4a4ade9bbcd, 9a29e16ebdcd41818aa407b2fe2343f8; no voters:
I20260812 05:19:36.131696 6957 raft_consensus.cc:2781] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:19:36.131783 6957 raft_consensus.cc:482] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:19:36.131821 6957 raft_consensus.cc:3037] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:36.132673 6957 raft_consensus.cc:504] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:36.132862 6957 leader_election.cc:288] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [CANDIDATE]: Term 1 election: Requested vote from peers 333d1783203e435aa7eca4a4ade9bbcd (127.18.166.193:45407), 682dbbcbde8640ff8ac3f262bfd52a1a (127.18.166.194:36543)
I20260812 05:19:36.133045 6606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2979fc1d9371411db2ede0d4e0a743d4" candidate_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "333d1783203e435aa7eca4a4ade9bbcd"
I20260812 05:19:36.133138 6606 raft_consensus.cc:3037] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:36.133136 6747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2979fc1d9371411db2ede0d4e0a743d4" candidate_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a"
I20260812 05:19:36.133236 6747 raft_consensus.cc:3037] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:36.133955 6606 raft_consensus.cc:2454] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a29e16ebdcd41818aa407b2fe2343f8 in term 1.
I20260812 05:19:36.134116 6747 raft_consensus.cc:2454] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a29e16ebdcd41818aa407b2fe2343f8 in term 1.
I20260812 05:19:36.134212 6818 leader_election.cc:302] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 333d1783203e435aa7eca4a4ade9bbcd, 9a29e16ebdcd41818aa407b2fe2343f8; no voters:
I20260812 05:19:36.134312 6957 raft_consensus.cc:2781] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:36.134524 6957 raft_consensus.cc:686] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9a29e16ebdcd41818aa407b2fe2343f8, State: Running, Role: LEADER
I20260812 05:19:36.134624 6957 consensus_queue.cc:237] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } }
I20260812 05:19:36.135439 6454 catalog_manager.cc:5495] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a29e16ebdcd41818aa407b2fe2343f8 (127.18.166.195). New cstate: current_term: 1 leader_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a29e16ebdcd41818aa407b2fe2343f8" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 35855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 } health_report { overall_health: UNKNOWN } } }
I20260812 05:19:36.156061 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:36.153834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.18.166.195:35855)
0812 05:19:36.153905 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:36.153907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:36.153924 (+ 17us) server_negotiation.cc:407] Connection header received
0812 05:19:36.153960 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:36.153961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:36.153980 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:36.154045 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:19:36.154172 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.154634 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.155347 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:36.155523 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:36.155571 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:36.155719 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:19:36.155801 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:19:36.155872 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:36.155915 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 05:19:36.155923 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:19:36.157960 6747 raft_consensus.cc:1261] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 1 FOLLOWER]: Refusing update from remote peer 9a29e16ebdcd41818aa407b2fe2343f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:36.157960 6606 raft_consensus.cc:1261] T 2979fc1d9371411db2ede0d4e0a743d4 P 333d1783203e435aa7eca4a4ade9bbcd [term 1 FOLLOWER]: Refusing update from remote peer 9a29e16ebdcd41818aa407b2fe2343f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:36.158361 6961 consensus_queue.cc:1035] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "333d1783203e435aa7eca4a4ade9bbcd" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 45407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:36.158506 6957 consensus_queue.cc:1035] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "682dbbcbde8640ff8ac3f262bfd52a1a" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:36.161527 6967 mvcc.cc:205] Tried to move back new op lower bound from 7317553054339874816 to 7317553054247972864. Current Snapshot: MvccSnapshot[applied={T|T < 7317553054339874816 or (T in {7317553054339874816})}]
I20260812 05:19:36.161633 6966 mvcc.cc:205] Tried to move back new op lower bound from 7317553054339874816 to 7317553054247972864. Current Snapshot: MvccSnapshot[applied={T|T < 7317553054339874816 or (T in {7317553054339874816})}]
I20260812 05:19:36.163151 6452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:36.163244 6452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:36.164038 6452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f47c75c357440cf8061ae317ee6e46c: Sending DeleteTablet for 3 replicas of tablet 2979fc1d9371411db2ede0d4e0a743d4
I20260812 05:19:36.164417 6865 tablet_service.cc:1514] Processing DeleteTablet for tablet 2979fc1d9371411db2ede0d4e0a743d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:36 UTC) from {username='slave'} at 127.0.0.1:46266
I20260812 05:19:36.164417 6583 tablet_service.cc:1514] Processing DeleteTablet for tablet 2979fc1d9371411db2ede0d4e0a743d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:36 UTC) from {username='slave'} at 127.0.0.1:60694
I20260812 05:19:36.164680 6976 tablet_replica.cc:330] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: stopping tablet replica
I20260812 05:19:36.164673 6725 tablet_service.cc:1514] Processing DeleteTablet for tablet 2979fc1d9371411db2ede0d4e0a743d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:36 UTC) from {username='slave'} at 127.0.0.1:43278
I20260812 05:19:36.164810 6976 raft_consensus.cc:2229] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:19:36.164923 6976 raft_consensus.cc:2258] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:36.165046 6977 tablet_replica.cc:330] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: stopping tablet replica
I20260812 05:19:36.165158 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6518
I20260812 05:19:36.165167 6977 raft_consensus.cc:2229] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:36.165268 6977 raft_consensus.cc:2258] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:36.167402 6977 ts_tablet_manager.cc:1905] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:36.168290 6976 ts_tablet_manager.cc:1905] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:36.169499 6977 ts_tablet_manager.cc:1918] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:36.169580 6977 log.cc:1196] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/2979fc1d9371411db2ede0d4e0a743d4
I20260812 05:19:36.169891 6977 ts_tablet_manager.cc:1939] T 2979fc1d9371411db2ede0d4e0a743d4 P 682dbbcbde8640ff8ac3f262bfd52a1a: Deleting consensus metadata
I20260812 05:19:36.169987 6976 ts_tablet_manager.cc:1918] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:36.170060 6976 log.cc:1196] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/2979fc1d9371411db2ede0d4e0a743d4
I20260812 05:19:36.170326 6976 ts_tablet_manager.cc:1939] T 2979fc1d9371411db2ede0d4e0a743d4 P 9a29e16ebdcd41818aa407b2fe2343f8: Deleting consensus metadata
I20260812 05:19:36.170375 6435 catalog_manager.cc:4841] TS 682dbbcbde8640ff8ac3f262bfd52a1a (127.18.166.194:36543): tablet 2979fc1d9371411db2ede0d4e0a743d4 (table test-table [id=773881d6fc874ce0bde9c70402a5f4c7]) successfully deleted
I20260812 05:19:36.170640 6436 catalog_manager.cc:4841] TS 9a29e16ebdcd41818aa407b2fe2343f8 (127.18.166.195:35855): tablet 2979fc1d9371411db2ede0d4e0a743d4 (table test-table [id=773881d6fc874ce0bde9c70402a5f4c7]) successfully deleted
W20260812 05:19:36.177162 6436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:45407 (error 108)
I20260812 05:19:36.177744 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6660
I20260812 05:19:36.177899 6947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:36.177739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:45407 (local address 127.0.0.1:60702)
0812 05:19:36.177809 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:36.177858 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:45407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 05:19:36.178972 6436 catalog_manager.cc:4584] TS 333d1783203e435aa7eca4a4ade9bbcd (127.18.166.193:45407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2979fc1d9371411db2ede0d4e0a743d4: Network error: Client connection negotiation failed: client connection to 127.18.166.193:45407: connect: Connection refused (error 111)
I20260812 05:19:36.184242 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6801
I20260812 05:19:36.191191 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6420
2026-08-12T05:19:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:36 dist-test-slave-srgv krb5kdc[6981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:36 dist-test-slave-srgv krb5kdc[6981](info): set up 2 sockets
Aug 12 05:19:36 dist-test-slave-srgv krb5kdc[6981](info): commencing operation
krb5kdc: starting...
W20260812 05:19:37.258279 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:19:37.381049 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39543
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:39543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:37.464003 6994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:37.464272 6994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:37.464313 6994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:37.465564 6994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:37.465626 6994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:37.465651 6994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:37.465680 6994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:37.465708 6994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:37.467202 6994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:39543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:37.467470 6994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:37.467703 6994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:37.472766 7000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:37.472766 7002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:37.472795 6994 server_base.cc:1034] running on GCE node
W20260812 05:19:37.472772 6999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:37.473203 6994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:37.473515 6994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:37.474675 6994 hybrid_clock.cc:648] HybridClock initialized: now 1786511977474666 us; error 35 us; skew 500 ppm
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:37.477262 6994 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:37.477794 6994 webserver.cc:460] Webserver started at http://127.18.166.254:40469/ using document root <none> and password file <none>
I20260812 05:19:37.477984 6994 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:37.478026 6994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:37.478121 6994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:37.479140 6994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "7b668b4835494a28ad5e93ee62b93f2c"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "316a1536edc2885bd8abb8dd003ce7d3"
server_key_iv: "d342d37da90250aed501d4f6368f12b2"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.479444 6994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "7b668b4835494a28ad5e93ee62b93f2c"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "316a1536edc2885bd8abb8dd003ce7d3"
server_key_iv: "d342d37da90250aed501d4f6368f12b2"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.480718 6994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:19:37.481439 7009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.481602 6994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:37.481690 6994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "7b668b4835494a28ad5e93ee62b93f2c"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "316a1536edc2885bd8abb8dd003ce7d3"
server_key_iv: "d342d37da90250aed501d4f6368f12b2"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.481755 6994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:37.515416 6994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:37.517865 6994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:37.517982 6994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:37.522091 6994 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:39543
I20260812 05:19:37.522171 7065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:39543 every 8 connection(s)
I20260812 05:19:37.522500 6994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:37.523450 7066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:37.525836 7066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c: Bootstrap starting.
I20260812 05:19:37.526418 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 6994
I20260812 05:19:37.526532 7066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:37.526556 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:37.526808 19099 external_mini_cluster.cc:1428] Setting key 1b403f1cc7e8a271f28192f72a16cdf9
I20260812 05:19:37.526889 7066 log.cc:824] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:37.527736 7066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c: No bootstrap required, opened a new log
I20260812 05:19:37.530121 7066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b668b4835494a28ad5e93ee62b93f2c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39543 } }
I20260812 05:19:37.530299 7066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:37.530335 7066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b668b4835494a28ad5e93ee62b93f2c, State: Initialized, Role: FOLLOWER
I20260812 05:19:37.530459 7066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b668b4835494a28ad5e93ee62b93f2c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39543 } }
I20260812 05:19:37.530535 7066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:37.530565 7066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:37.530602 7066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:37.531239 7066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b668b4835494a28ad5e93ee62b93f2c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39543 } }
I20260812 05:19:37.531327 7066 leader_election.cc:302] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b668b4835494a28ad5e93ee62b93f2c; no voters:
I20260812 05:19:37.531530 7066 leader_election.cc:288] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:37.531577 7071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:37.531749 7071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [term 1 LEADER]: Becoming Leader. State: Replica: 7b668b4835494a28ad5e93ee62b93f2c, State: Running, Role: LEADER
I20260812 05:19:37.531804 7066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:37.531865 7071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b668b4835494a28ad5e93ee62b93f2c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39543 } }
I20260812 05:19:37.532191 7072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b668b4835494a28ad5e93ee62b93f2c. Latest consensus state: current_term: 1 leader_uuid: "7b668b4835494a28ad5e93ee62b93f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b668b4835494a28ad5e93ee62b93f2c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39543 } } }
I20260812 05:19:37.532197 7073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b668b4835494a28ad5e93ee62b93f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b668b4835494a28ad5e93ee62b93f2c" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39543 } } }
I20260812 05:19:37.532279 7072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:37.532298 7073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:37.532501 7075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:37.533006 7075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:37.534658 7075 catalog_manager.cc:1347] Generated new cluster ID: 83e917fde65f4cc2ac6f04241aaa2bf4
I20260812 05:19:37.534710 7075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:37.538359 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:37.527339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.18.166.254:39543)
0812 05:19:37.527484 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:37.527493 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:37.527530 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:19:37.528029 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:37.528036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:37.528095 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:37.528209 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:37.529023 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.530030 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.531054 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.531575 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.531691 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:37.534743 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:37.534746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:37.534746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:37.534757 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:37.536663 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:37.537073 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:37.537076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:37.537076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:37.537175 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:37.537395 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:37.537395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:37.537396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:37.537712 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:37.537867 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:37.538017 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 05:19:37.538149 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260812 05:19:37.539494 7075 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:37.539987 7075 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:37.550653 7075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c: Generated new TSK 0
I20260812 05:19:37.550861 7075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.166.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
Entry for principal kudu/127.18.166.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab.
I20260812 05:19:37.578492 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39543
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:37.659852 7093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:37.660086 7093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:37.660127 7093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:37.661276 7093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:37.661336 7093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:37.661387 7093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:37.662864 7093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.193.keytab
--principal=kudu/127.18.166.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:37.663115 7093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:37.663378 7093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:37.668402 7101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:37.668413 7099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:37.668421 7098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:37.668726 7093 server_base.cc:1034] running on GCE node
I20260812 05:19:37.668890 7093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:37.669137 7093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:37.670301 7093 hybrid_clock.cc:648] HybridClock initialized: now 1786511977670292 us; error 38 us; skew 500 ppm
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:37.672919 7093 init.cc:377] Logged in from keytab as kudu/127.18.166.193@KRBTEST.COM (short username kudu)
I20260812 05:19:37.673317 7093 webserver.cc:460] Webserver started at http://127.18.166.193:34811/ using document root <none> and password file <none>
I20260812 05:19:37.673571 7093 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:37.673619 7093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:37.673727 7093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:37.674742 7093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "f8d295148d9a4080ac1a471affbd3f30"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "c8019f6b4f0dab468cc93ffd9e51d8d9"
server_key_iv: "a07e914e7b315b7e270178ab78e0a759"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.675045 7093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "f8d295148d9a4080ac1a471affbd3f30"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "c8019f6b4f0dab468cc93ffd9e51d8d9"
server_key_iv: "a07e914e7b315b7e270178ab78e0a759"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.676306 7093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:37.676965 7110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.677119 7093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:37.677243 7093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "f8d295148d9a4080ac1a471affbd3f30"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "c8019f6b4f0dab468cc93ffd9e51d8d9"
server_key_iv: "a07e914e7b315b7e270178ab78e0a759"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.677325 7093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:37.694761 7093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:37.696660 7093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:37.696794 7093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:37.696992 7093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:37.697324 7093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:37.697399 7093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.697475 7093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:37.697504 7093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.703269 7093 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:35895
I20260812 05:19:37.703423 7230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:35895 every 8 connection(s)
I20260812 05:19:37.703692 7093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:19:37.703994 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7093
I20260812 05:19:37.704133 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:19:37.704373 19099 external_mini_cluster.cc:1428] Setting key e22bb5416527816ca6e315d7b47bf2f3
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.193@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.194@KRBTEST.COM; defaulting to no policy
I20260812 05:19:37.715145 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:37.704767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:54011 (local address 127.18.166.254:39543)
0812 05:19:37.704834 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:37.704836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:37.705059 (+ 223us) server_negotiation.cc:407] Connection header received
0812 05:19:37.705810 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:37.705811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:37.705820 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:37.705858 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:37.707209 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.707724 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.709306 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.709586 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.709644 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:37.712233 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:37.712235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:37.712235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:37.712242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:37.713721 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:37.714190 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:37.714191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:37.714192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:37.714257 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:37.714509 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:37.714510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:37.714510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:37.714629 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:37.714709 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:37.714993 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 05:19:37.715076 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:19:37.715411 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:37.704758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39543 (local address 127.18.166.193:54011)
0812 05:19:37.704943 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:37.704985 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:37.705708 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:37.705939 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:37.705944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:37.706808 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:37.707113 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:37.707120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.707825 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.707826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:37.709202 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:37.709206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.709676 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.709678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:37.709853 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:37.710422 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:37.710434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:37.712113 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:37.713817 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:37.713824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:37.713827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:37.714103 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:37.714333 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:37.714334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:37.714334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:37.714446 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:37.714699 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:37.714707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:37.714857 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 05:19:37.714970 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 05:19:37.715142 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260812 05:19:37.715930 7232 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39543
I20260812 05:19:37.716048 7232 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:37.716269 7232 heartbeater.cc:511] Master 127.18.166.254:39543 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.194@KRBTEST.COM" created.
I20260812 05:19:37.716871 7029 ts_manager.cc:194] Registered new tserver with Master: f8d295148d9a4080ac1a471affbd3f30 (127.18.166.193:35895)
I20260812 05:19:37.717555 7029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.193@KRBTEST.COM'} at 127.18.166.193:54011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
Entry for principal kudu/127.18.166.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab.
I20260812 05:19:37.731664 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39543
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:37.813745 7237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:37.813952 7237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:37.813977 7237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:37.815032 7237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:37.815068 7237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:37.815092 7237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:37.816569 7237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.194.keytab
--principal=kudu/127.18.166.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:37.816763 7237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:37.816949 7237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:37.822088 7242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:37.822113 7245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:37.822149 7237 server_base.cc:1034] running on GCE node
W20260812 05:19:37.822095 7243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:37.822487 7237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:37.822788 7237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:37.823959 7237 hybrid_clock.cc:648] HybridClock initialized: now 1786511977823947 us; error 25 us; skew 500 ppm
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:37.826735 7237 init.cc:377] Logged in from keytab as kudu/127.18.166.194@KRBTEST.COM (short username kudu)
I20260812 05:19:37.827244 7237 webserver.cc:460] Webserver started at http://127.18.166.194:41983/ using document root <none> and password file <none>
I20260812 05:19:37.827520 7237 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:37.827592 7237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:37.827749 7237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:37.828825 7237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "67e30d977b27c881a2d622ef3db3e424"
server_key_iv: "0e825cf2e2c8127d5b9dc16aa6356892"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.829207 7237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "67e30d977b27c881a2d622ef3db3e424"
server_key_iv: "0e825cf2e2c8127d5b9dc16aa6356892"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.830657 7237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:37.831435 7252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.831614 7237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.831722 7237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "67e30d977b27c881a2d622ef3db3e424"
server_key_iv: "0e825cf2e2c8127d5b9dc16aa6356892"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.831799 7237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:37.845094 7237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:37.847653 7237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:37.847833 7237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:37.848084 7237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:37.848410 7237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:37.848465 7237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.848500 7237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:37.848546 7237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.854707 7237 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:45391
I20260812 05:19:37.854774 7371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:45391 every 8 connection(s)
I20260812 05:19:37.855103 7237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:19:37.857041 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7237
I20260812 05:19:37.857141 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:19:37.857360 19099 external_mini_cluster.cc:1428] Setting key 4dc927bd510de2ab88fc08c51799ce0e
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.194@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:37.865370 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:37.855863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:34033 (local address 127.18.166.254:39543)
0812 05:19:37.855944 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:37.855946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:37.856161 (+ 215us) server_negotiation.cc:407] Connection header received
0812 05:19:37.856655 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:37.856656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:37.856666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:37.856723 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:37.857922 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.858561 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.859540 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.859817 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.859876 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:37.862316 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:37.862318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:37.862318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:37.862327 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:37.863796 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:37.864278 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:37.864280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:37.864281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:37.864345 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:37.864615 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:37.864616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:37.864616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:37.864778 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:37.864851 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:37.865230 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 05:19:37.865303 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:19:37.865653 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:37.855835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39543 (local address 127.18.166.194:34033)
0812 05:19:37.856025 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:37.856072 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:37.856565 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:37.856786 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:37.856789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:37.857534 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:37.857831 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:37.857838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.858658 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.858659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:37.859446 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:37.859448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:37.859908 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:37.859909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:37.860069 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:37.860652 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:37.860663 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:37.862168 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:37.863904 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:37.863912 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:37.863914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:37.864181 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:37.864416 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:37.864417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:37.864418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:37.864539 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:37.864869 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:37.864877 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:37.865056 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 05:19:37.865192 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 05:19:37.865376 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260812 05:19:37.866142 7372 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39543
I20260812 05:19:37.866261 7372 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:37.866473 7372 heartbeater.cc:511] Master 127.18.166.254:39543 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.166.195@KRBTEST.COM; defaulting to no policy
I20260812 05:19:37.866900 7029 ts_manager.cc:194] Registered new tserver with Master: 86e95addbe1b4ce7a7cde66dfd9bd24c (127.18.166.194:45391)
I20260812 05:19:37.867278 7029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.194@KRBTEST.COM'} at 127.18.166.194:34033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
Entry for principal kudu/127.18.166.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab.
I20260812 05:19:37.884409 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39543
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:37.973397 7377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:37.973691 7377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:37.973716 7377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:37.974893 7377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:37.974938 7377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:37.974965 7377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:37.976436 7377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.195.keytab
--principal=kudu/127.18.166.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:39543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:37.976686 7377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:37.976948 7377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:37.982204 7383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:37.982218 7382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:37.982271 7385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:37.982522 7377 server_base.cc:1034] running on GCE node
I20260812 05:19:37.982735 7377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:37.982993 7377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:37.984155 7377 hybrid_clock.cc:648] HybridClock initialized: now 1786511977984145 us; error 27 us; skew 500 ppm
Aug 12 05:19:37 dist-test-slave-srgv krb5kdc[6981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:37.986831 7377 init.cc:377] Logged in from keytab as kudu/127.18.166.195@KRBTEST.COM (short username kudu)
I20260812 05:19:37.987221 7377 webserver.cc:460] Webserver started at http://127.18.166.195:40551/ using document root <none> and password file <none>
I20260812 05:19:37.987401 7377 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:37.987442 7377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:37.987534 7377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:37.988528 7377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "ead2b607405e4daba1e0609299313d7d"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "7a0e93328a0de3f29de902c589a7bb76"
server_key_iv: "ccceb1ac51968230867a73bee61fe8eb"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.988914 7377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "ead2b607405e4daba1e0609299313d7d"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "7a0e93328a0de3f29de902c589a7bb76"
server_key_iv: "ccceb1ac51968230867a73bee61fe8eb"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.990327 7377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:19:37.990996 7392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:37.991169 7377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:37.991237 7377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "ead2b607405e4daba1e0609299313d7d"
format_stamp: "Formatted at 2026-08-12 05:19:37 on dist-test-slave-srgv"
server_key: "7a0e93328a0de3f29de902c589a7bb76"
server_key_iv: "ccceb1ac51968230867a73bee61fe8eb"
server_key_version: "encryptionkey@0"
I20260812 05:19:37.991291 7377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:38.001343 7377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:38.003692 7377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:38.003813 7377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:38.004000 7377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:38.004333 7377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:38.004369 7377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.004410 7377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:38.004436 7377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.010279 7377 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:38515
I20260812 05:19:38.010380 7509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:38515 every 8 connection(s)
I20260812 05:19:38.010701 7377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:19:38 dist-test-slave-srgv krb5kdc[6981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.195@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:38.020334 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7377
I20260812 05:19:38.020352 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.011468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:36071 (local address 127.18.166.254:39543)
0812 05:19:38.011539 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.011542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.011756 (+ 214us) server_negotiation.cc:407] Connection header received
0812 05:19:38.012287 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.012289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.012301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.012364 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.013314 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.013887 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.014881 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.015096 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.015184 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.017444 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.017445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.017446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:38.017454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.018985 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:38.019381 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:38.019382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:38.019383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:38.019444 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:38.019708 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:38.019709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:38.019709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:38.019825 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.019893 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.020171 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.020256 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:19:38.020476 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:19:38.020552 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.011441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:39543 (local address 127.18.166.195:36071)
0812 05:19:38.011628 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.011664 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.012175 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.012439 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.012442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.013014 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:38.013209 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.013214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.013982 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.013984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.014781 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.014783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.015195 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.015196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.015320 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.015862 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:19:38.015870 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:38.017315 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:38.019067 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:38.019074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:38.019076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:38.019302 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:38.019515 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:19:38.019516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:19:38.019516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:19:38.019634 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:19:38.019897 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:38.019904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:38.020039 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.020147 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.020312 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":120,"threads_started":1}
I20260812 05:19:38.020685 19099 external_mini_cluster.cc:1428] Setting key 5024b918a027c9d8b7c328efa38d915c
I20260812 05:19:38.021044 7510 heartbeater.cc:347] Connected to a master server at 127.18.166.254:39543
I20260812 05:19:38.021152 7510 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:38.021363 7510 heartbeater.cc:511] Master 127.18.166.254:39543 requested a full tablet report, sending...
I20260812 05:19:38.021812 7029 ts_manager.cc:194] Registered new tserver with Master: ead2b607405e4daba1e0609299313d7d (127.18.166.195:38515)
I20260812 05:19:38.022149 7029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.166.195@KRBTEST.COM'} at 127.18.166.195:36071
I20260812 05:19:38.032514 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:19:38.040648 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.034108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.18.166.254:39543)
0812 05:19:38.034171 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.034174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.034274 (+ 100us) server_negotiation.cc:407] Connection header received
0812 05:19:38.034353 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.034354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.034365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.034415 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.035095 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.035589 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.036376 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.036580 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.036655 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.037995 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.037997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.037997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:38.038004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.039458 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:38.039826 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:38.039827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:38.039827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:38.039887 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:38.040094 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:38.040095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:38.040096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:38.040249 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.040359 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.040459 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.040544 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:19:38.042362 7029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:19:38.043983 7029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:19:38.050321 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.046267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35895 (local address 127.0.0.1:53102)
0812 05:19:38.046408 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.046421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.046483 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.046854 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.046855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.046857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.046886 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.046887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.048220 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.048221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.048993 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.048995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.049971 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.049972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.050082 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.050098 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.050175 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.050231 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":39,"threads_started":1}
I20260812 05:19:38.050604 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.046400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.18.166.193:35895)
0812 05:19:38.046635 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.046639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.046649 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:38.046686 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.046688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.046701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.046773 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.046952 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.048103 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.049100 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.049876 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.049965 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.050031 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.050465 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.050523 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
I20260812 05:19:38.050614 7518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.046042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:38515 (local address 127.0.0.1:43548)
0812 05:19:38.046162 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.046203 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.046278 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.046736 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.046739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.046742 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.046815 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.046817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.048206 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.048207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.049331 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.049333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.050384 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.050385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.050449 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.050454 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.050501 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.050540 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260812 05:19:38.050736 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.046047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:45391 (local address 127.0.0.1:60566)
0812 05:19:38.046379 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.046388 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.046426 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.046803 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.046804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.046806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.046874 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.046876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.047953 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.047955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.049214 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.049218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.050127 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.050128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.050200 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.050204 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.050271 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.050317 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":69,"threads_started":1}
I20260812 05:19:38.050662 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.046383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.18.166.195:38515)
0812 05:19:38.046561 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.046566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.046576 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:38.046617 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.046619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.046631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.046722 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.046884 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.048092 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.049518 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.050311 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.050374 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.050428 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.050525 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.050572 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260812 05:19:38.051087 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.046175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.18.166.194:45391)
0812 05:19:38.046613 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.046618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.046629 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:19:38.046673 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.046676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.046692 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.047002 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.047090 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.047850 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.049312 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.050057 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.050125 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.050171 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.050286 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.050326 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":134,"threads_started":1}
I20260812 05:19:38.051400 7162 tablet_service.cc:1467] Processing CreateTablet for tablet 3599ee050fe547c7a4184480f6a2bb94 (DEFAULT_TABLE table=test-table [id=7f84ba7888ca4d0eb59c3eb07ad5a8db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:38.051791 7162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3599ee050fe547c7a4184480f6a2bb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.051705 7303 tablet_service.cc:1467] Processing CreateTablet for tablet 3599ee050fe547c7a4184480f6a2bb94 (DEFAULT_TABLE table=test-table [id=7f84ba7888ca4d0eb59c3eb07ad5a8db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:38.052186 7303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3599ee050fe547c7a4184480f6a2bb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.053885 7441 tablet_service.cc:1467] Processing CreateTablet for tablet 3599ee050fe547c7a4184480f6a2bb94 (DEFAULT_TABLE table=test-table [id=7f84ba7888ca4d0eb59c3eb07ad5a8db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:38.054214 7441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3599ee050fe547c7a4184480f6a2bb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.054898 7524 tablet_bootstrap.cc:492] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Bootstrap starting.
I20260812 05:19:38.055349 7525 tablet_bootstrap.cc:492] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: Bootstrap starting.
I20260812 05:19:38.055667 7524 tablet_bootstrap.cc:654] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.056017 7525 tablet_bootstrap.cc:654] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.056309 7524 log.cc:824] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.056600 7525 log.cc:824] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.057183 7528 tablet_bootstrap.cc:492] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Bootstrap starting.
I20260812 05:19:38.057334 7524 tablet_bootstrap.cc:492] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: No bootstrap required, opened a new log
I20260812 05:19:38.057430 7524 ts_tablet_manager.cc:1397] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:19:38.057853 7528 tablet_bootstrap.cc:654] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.058117 7525 tablet_bootstrap.cc:492] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: No bootstrap required, opened a new log
I20260812 05:19:38.058197 7525 ts_tablet_manager.cc:1397] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:19:38.058446 7528 log.cc:824] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.059227 7528 tablet_bootstrap.cc:492] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: No bootstrap required, opened a new log
I20260812 05:19:38.059304 7528 ts_tablet_manager.cc:1397] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:19:38.059523 7524 raft_consensus.cc:348] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.059692 7524 raft_consensus.cc:374] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.059739 7524 raft_consensus.cc:729] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e95addbe1b4ce7a7cde66dfd9bd24c, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.059846 7524 consensus_queue.cc:260] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.060128 7524 ts_tablet_manager.cc:1428] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:19:38.060139 7372 heartbeater.cc:503] Master 127.18.166.254:39543 was elected leader, sending a full tablet report...
I20260812 05:19:38.060303 7525 raft_consensus.cc:348] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.060488 7525 raft_consensus.cc:374] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.060531 7525 raft_consensus.cc:729] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d295148d9a4080ac1a471affbd3f30, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.060648 7525 consensus_queue.cc:260] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.060942 7232 heartbeater.cc:503] Master 127.18.166.254:39543 was elected leader, sending a full tablet report...
I20260812 05:19:38.060925 7525 ts_tablet_manager.cc:1428] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:19:38.061298 7528 raft_consensus.cc:348] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.061519 7528 raft_consensus.cc:374] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.061576 7528 raft_consensus.cc:729] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead2b607405e4daba1e0609299313d7d, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.061693 7528 consensus_queue.cc:260] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.061992 7528 ts_tablet_manager.cc:1428] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:19:38.062006 7510 heartbeater.cc:503] Master 127.18.166.254:39543 was elected leader, sending a full tablet report...
W20260812 05:19:38.105785 7373 tablet.cc:2369] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:19:38.153469 7530 raft_consensus.cc:482] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:38.153604 7530 raft_consensus.cc:504] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.153925 7530 leader_election.cc:288] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ead2b607405e4daba1e0609299313d7d (127.18.166.195:38515), f8d295148d9a4080ac1a471affbd3f30 (127.18.166.193:35895)
I20260812 05:19:38.156773 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.154280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:60415 (local address 127.18.166.195:38515)
0812 05:19:38.154334 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.154336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.154348 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:19:38.154381 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.154381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.154391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.154465 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.154645 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.155113 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.156046 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.156412 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.156498 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.156508 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.156662 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.156700 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:19:38.156772 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.154145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:38515 (local address 127.18.166.194:60415)
0812 05:19:38.154210 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.154223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.154283 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.154488 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.154490 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.154492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.154564 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.154567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.155222 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.155223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.155943 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.155945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.156496 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.156497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.156581 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.156584 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.156652 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.156694 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:19:38.157063 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.154391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:53211 (local address 127.18.166.193:35895)
0812 05:19:38.154471 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.154473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.154603 (+ 130us) server_negotiation.cc:407] Connection header received
0812 05:19:38.154668 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.154669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.154681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.154762 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.154924 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.155392 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.156315 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.156706 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.156795 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.156803 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.156949 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.156994 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:19:38.157061 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.154304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35895 (local address 127.18.166.194:53211)
0812 05:19:38.154531 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.154540 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.154609 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.154764 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.154765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.154768 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.154847 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.154849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.155483 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.155484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.156216 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.156218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.156797 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.156798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.156873 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.156878 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.156939 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.156987 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":148,"threads_started":1}
I20260812 05:19:38.157155 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3599ee050fe547c7a4184480f6a2bb94" candidate_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead2b607405e4daba1e0609299313d7d" is_pre_election: true
I20260812 05:19:38.157303 7461 raft_consensus.cc:2454] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86e95addbe1b4ce7a7cde66dfd9bd24c in term 0.
I20260812 05:19:38.157544 7253 leader_election.cc:302] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86e95addbe1b4ce7a7cde66dfd9bd24c, ead2b607405e4daba1e0609299313d7d; no voters:
I20260812 05:19:38.157704 7530 raft_consensus.cc:2781] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:19:38.157809 7530 raft_consensus.cc:482] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:19:38.157850 7530 raft_consensus.cc:3037] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.157907 7183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3599ee050fe547c7a4184480f6a2bb94" candidate_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d295148d9a4080ac1a471affbd3f30" is_pre_election: true
I20260812 05:19:38.158057 7183 raft_consensus.cc:2454] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86e95addbe1b4ce7a7cde66dfd9bd24c in term 0.
I20260812 05:19:38.158663 7530 raft_consensus.cc:504] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.158813 7530 leader_election.cc:288] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [CANDIDATE]: Term 1 election: Requested vote from peers ead2b607405e4daba1e0609299313d7d (127.18.166.195:38515), f8d295148d9a4080ac1a471affbd3f30 (127.18.166.193:35895)
I20260812 05:19:38.158991 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3599ee050fe547c7a4184480f6a2bb94" candidate_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead2b607405e4daba1e0609299313d7d"
I20260812 05:19:38.159025 7183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3599ee050fe547c7a4184480f6a2bb94" candidate_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d295148d9a4080ac1a471affbd3f30"
I20260812 05:19:38.159075 7461 raft_consensus.cc:3037] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.159117 7183 raft_consensus.cc:3037] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.159837 7183 raft_consensus.cc:2454] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86e95addbe1b4ce7a7cde66dfd9bd24c in term 1.
I20260812 05:19:38.159840 7461 raft_consensus.cc:2454] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86e95addbe1b4ce7a7cde66dfd9bd24c in term 1.
I20260812 05:19:38.160056 7253 leader_election.cc:302] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86e95addbe1b4ce7a7cde66dfd9bd24c, f8d295148d9a4080ac1a471affbd3f30; no voters:
I20260812 05:19:38.160172 7530 raft_consensus.cc:2781] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:38.160352 7530 raft_consensus.cc:686] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 1 LEADER]: Becoming Leader. State: Replica: 86e95addbe1b4ce7a7cde66dfd9bd24c, State: Running, Role: LEADER
I20260812 05:19:38.160465 7530 consensus_queue.cc:237] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } }
I20260812 05:19:38.161384 7028 catalog_manager.cc:5495] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c reported cstate change: term changed from 0 to 1, leader changed from <none> to 86e95addbe1b4ce7a7cde66dfd9bd24c (127.18.166.194). New cstate: current_term: 1 leader_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86e95addbe1b4ce7a7cde66dfd9bd24c" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 45391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 } health_report { overall_health: UNKNOWN } } }
I20260812 05:19:38.166491 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.164329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.18.166.194:45391)
0812 05:19:38.164378 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.164380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.164388 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:19:38.164461 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.164462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.164478 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.164534 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:19:38.164612 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.165055 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.165853 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.166033 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.166080 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.166217 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:19:38.166309 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:19:38.166372 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.166405 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.166423 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:19:38.168275 7183 raft_consensus.cc:1261] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 1 FOLLOWER]: Refusing update from remote peer 86e95addbe1b4ce7a7cde66dfd9bd24c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:38.168298 7461 raft_consensus.cc:1261] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 1 FOLLOWER]: Refusing update from remote peer 86e95addbe1b4ce7a7cde66dfd9bd24c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:38.168622 7530 consensus_queue.cc:1035] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead2b607405e4daba1e0609299313d7d" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:38.168818 7534 consensus_queue.cc:1035] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8d295148d9a4080ac1a471affbd3f30" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:38.171010 7537 mvcc.cc:205] Tried to move back new op lower bound from 7317553062574780416 to 7317553062545924096. Current Snapshot: MvccSnapshot[applied={T|T < 7317553062574780416}]
I20260812 05:19:38.171180 7538 mvcc.cc:205] Tried to move back new op lower bound from 7317553062574780416 to 7317553062545924096. Current Snapshot: MvccSnapshot[applied={T|T < 7317553062574780416}]
I20260812 05:19:38.174671 7028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:38.174757 7028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:38.175598 7028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b668b4835494a28ad5e93ee62b93f2c: Sending DeleteTablet for 3 replicas of tablet 3599ee050fe547c7a4184480f6a2bb94
I20260812 05:19:38.175956 7441 tablet_service.cc:1514] Processing DeleteTablet for tablet 3599ee050fe547c7a4184480f6a2bb94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:43548
I20260812 05:19:38.175980 7162 tablet_service.cc:1514] Processing DeleteTablet for tablet 3599ee050fe547c7a4184480f6a2bb94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:53102
I20260812 05:19:38.176208 7548 tablet_replica.cc:330] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: stopping tablet replica
I20260812 05:19:38.176213 7303 tablet_service.cc:1514] Processing DeleteTablet for tablet 3599ee050fe547c7a4184480f6a2bb94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:38 UTC) from {username='kudu', principal='kudu/127.18.166.254@KRBTEST.COM'} at 127.0.0.1:60566
I20260812 05:19:38.176354 7548 raft_consensus.cc:2229] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:38.176441 7549 tablet_replica.cc:330] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: stopping tablet replica
I20260812 05:19:38.176512 7548 raft_consensus.cc:2258] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:38.176537 7549 raft_consensus.cc:2229] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:19:38.176663 7549 raft_consensus.cc:2258] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:38.176899 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7093
I20260812 05:19:38.177109 7548 ts_tablet_manager.cc:1905] T 3599ee050fe547c7a4184480f6a2bb94 P f8d295148d9a4080ac1a471affbd3f30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:38.177852 7547 tablet_replica.cc:330] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: stopping tablet replica
I20260812 05:19:38.177986 7547 raft_consensus.cc:2229] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:38.178117 7547 raft_consensus.cc:2258] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:38.179226 7549 ts_tablet_manager.cc:1905] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:38.181344 7549 ts_tablet_manager.cc:1918] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:38.181478 7549 log.cc:1196] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/3599ee050fe547c7a4184480f6a2bb94
I20260812 05:19:38.181756 7549 ts_tablet_manager.cc:1939] T 3599ee050fe547c7a4184480f6a2bb94 P 86e95addbe1b4ce7a7cde66dfd9bd24c: Deleting consensus metadata
I20260812 05:19:38.182173 7011 catalog_manager.cc:4841] TS 86e95addbe1b4ce7a7cde66dfd9bd24c (127.18.166.194:45391): tablet 3599ee050fe547c7a4184480f6a2bb94 (table test-table [id=7f84ba7888ca4d0eb59c3eb07ad5a8db]) successfully deleted
W20260812 05:19:38.184900 7010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:35895 (error 108)
I20260812 05:19:38.185307 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7237
I20260812 05:19:38.185438 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.185282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:35895 (local address 127.0.0.1:53106)
0812 05:19:38.185338 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.185380 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 05:19:38.186661 7010 catalog_manager.cc:4584] TS f8d295148d9a4080ac1a471affbd3f30 (127.18.166.193:35895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3599ee050fe547c7a4184480f6a2bb94: Network error: Client connection negotiation failed: client connection to 127.18.166.193:35895: connect: Connection refused (error 111)
I20260812 05:19:38.191627 7547 ts_tablet_manager.cc:1905] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:38.193729 7547 ts_tablet_manager.cc:1918] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:38.193817 7547 log.cc:1196] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/3599ee050fe547c7a4184480f6a2bb94
I20260812 05:19:38.194108 7547 ts_tablet_manager.cc:1939] T 3599ee050fe547c7a4184480f6a2bb94 P ead2b607405e4daba1e0609299313d7d: Deleting consensus metadata
I20260812 05:19:38.194748 7010 catalog_manager.cc:4841] TS ead2b607405e4daba1e0609299313d7d (127.18.166.195:38515): tablet 3599ee050fe547c7a4184480f6a2bb94 (table test-table [id=7f84ba7888ca4d0eb59c3eb07ad5a8db]) successfully deleted
I20260812 05:19:38.195552 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7377
I20260812 05:19:38.205623 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 6994
2026-08-12T05:19:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T05:19:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:38Z Disabled control of system clock
I20260812 05:19:38.235268 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36995
--webserver_interface=127.18.166.254
--webserver_port=0
--builtin_ntp_servers=127.18.166.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:36995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:38.317029 7554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:38.317262 7554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:38.317287 7554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:38.318509 7554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:38.318567 7554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:38.318580 7554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:38.318593 7554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:38.318605 7554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:38.319928 7554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:36995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:38.320133 7554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:38.320380 7554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:38.325265 7560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.325315 7559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.325271 7562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:38.325503 7554 server_base.cc:1034] running on GCE node
I20260812 05:19:38.325775 7554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:38.326025 7554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:38.327183 7554 hybrid_clock.cc:648] HybridClock initialized: now 1786511978327158 us; error 39 us; skew 500 ppm
I20260812 05:19:38.328549 7554 webserver.cc:460] Webserver started at http://127.18.166.254:45527/ using document root <none> and password file <none>
I20260812 05:19:38.328821 7554 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:38.328869 7554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:38.329035 7554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:38.330176 7554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "ebd24e1334ce4627866ba61029b8225b"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "3dea2633c9c534926a5d71a85822ceca"
server_key_iv: "4457a584aa68e7f8a518c597d21b2d66"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.330547 7554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "ebd24e1334ce4627866ba61029b8225b"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "3dea2633c9c534926a5d71a85822ceca"
server_key_iv: "4457a584aa68e7f8a518c597d21b2d66"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.332002 7554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:19:38.332755 7569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.332952 7554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:38.333045 7554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "ebd24e1334ce4627866ba61029b8225b"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "3dea2633c9c534926a5d71a85822ceca"
server_key_iv: "4457a584aa68e7f8a518c597d21b2d66"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.333127 7554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:38.344269 7554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:38.344604 7554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:38.344753 7554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:38.348953 7554 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:36995
I20260812 05:19:38.348989 7625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:36995 every 8 connection(s)
I20260812 05:19:38.349350 7554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:38.349704 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7554
I20260812 05:19:38.349818 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:38.349992 19099 external_mini_cluster.cc:1428] Setting key 17c00c19e3ef1eb840775b827208e4e0
I20260812 05:19:38.350615 7626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.354019 7626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b: Bootstrap starting.
I20260812 05:19:38.354765 7626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.355105 7626 log.cc:824] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.356016 7626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b: No bootstrap required, opened a new log
I20260812 05:19:38.356575 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.350543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.18.166.254:36995)
0812 05:19:38.350782 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.350792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.350824 (+ 32us) server_negotiation.cc:407] Connection header received
0812 05:19:38.351437 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.351446 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.351529 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.351687 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.351880 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.353222 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.354075 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.354692 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.354826 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.355674 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.355679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.355679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:38.355891 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.355937 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:38.355952 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.356025 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.356264 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.356347 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260812 05:19:38.358230 7626 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd24e1334ce4627866ba61029b8225b" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36995 } }
I20260812 05:19:38.358364 7626 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.358388 7626 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd24e1334ce4627866ba61029b8225b, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.358474 7626 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd24e1334ce4627866ba61029b8225b" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36995 } }
I20260812 05:19:38.358525 7626 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:38.358546 7626 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:38.358572 7626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.359118 7626 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd24e1334ce4627866ba61029b8225b" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36995 } }
I20260812 05:19:38.359210 7626 leader_election.cc:302] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebd24e1334ce4627866ba61029b8225b; no voters:
I20260812 05:19:38.359346 7626 leader_election.cc:288] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:38.359419 7631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:38.359597 7631 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [term 1 LEADER]: Becoming Leader. State: Replica: ebd24e1334ce4627866ba61029b8225b, State: Running, Role: LEADER
I20260812 05:19:38.359624 7626 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:38.359692 7631 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd24e1334ce4627866ba61029b8225b" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36995 } }
I20260812 05:19:38.360083 7633 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebd24e1334ce4627866ba61029b8225b. Latest consensus state: current_term: 1 leader_uuid: "ebd24e1334ce4627866ba61029b8225b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd24e1334ce4627866ba61029b8225b" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36995 } } }
I20260812 05:19:38.360196 7633 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:38.360074 7632 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebd24e1334ce4627866ba61029b8225b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd24e1334ce4627866ba61029b8225b" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36995 } } }
I20260812 05:19:38.360476 7640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:38.360411 7632 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:38.360978 7640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:38.362617 7640 catalog_manager.cc:1347] Generated new cluster ID: c17f0009fcd64e1497376f8edc75bbba
I20260812 05:19:38.362684 7640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:38.371830 7640 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:38.372382 7640 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:38.381022 7640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b: Generated new TSK 0
I20260812 05:19:38.381253 7640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:38.390564 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.193:0
--local_ip_for_outbound_sockets=127.18.166.193
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36995
--builtin_ntp_servers=127.18.166.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:38.476598 7652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:38.476823 7652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:38.476847 7652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:38.478121 7652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:38.478169 7652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:38.478197 7652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.193
I20260812 05:19:38.479570 7652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.166.193
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.193
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:38.479832 7652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:38.480166 7652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:38.486230 7658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.486274 7657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.486254 7661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:38.486788 7652 server_base.cc:1034] running on GCE node
I20260812 05:19:38.487043 7652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:38.487303 7652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:38.488474 7652 hybrid_clock.cc:648] HybridClock initialized: now 1786511978488462 us; error 32 us; skew 500 ppm
I20260812 05:19:38.489923 7652 webserver.cc:460] Webserver started at http://127.18.166.193:42545/ using document root <none> and password file <none>
I20260812 05:19:38.490185 7652 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:38.490257 7652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:38.490410 7652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:38.491505 7652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data/instance:
uuid: "5a79a09c605440218952bdd3ef9a4883"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "11771d217ad8cb8f522ecd8e01256d76"
server_key_iv: "4e2f55ea778aeaa768e818621d47dd0d"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.491883 7652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance:
uuid: "5a79a09c605440218952bdd3ef9a4883"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "11771d217ad8cb8f522ecd8e01256d76"
server_key_iv: "4e2f55ea778aeaa768e818621d47dd0d"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.493289 7652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:19:38.494169 7667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.494403 7652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:38.494505 7652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal
uuid: "5a79a09c605440218952bdd3ef9a4883"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "11771d217ad8cb8f522ecd8e01256d76"
server_key_iv: "4e2f55ea778aeaa768e818621d47dd0d"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.494577 7652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:38.509328 7652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:38.509712 7652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:38.509833 7652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:38.510046 7652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:38.510421 7652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:38.510483 7652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.510529 7652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:38.510557 7652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.516412 7652 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.193:42295
I20260812 05:19:38.516528 7785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.193:42295 every 8 connection(s)
I20260812 05:19:38.516878 7652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/data/info.pb
I20260812 05:19:38.522558 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.517732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.193:54813 (local address 127.18.166.254:36995)
0812 05:19:38.517779 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.517781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.518060 (+ 279us) server_negotiation.cc:407] Connection header received
0812 05:19:38.518650 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.518651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.518671 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.518743 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.519445 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.519983 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.520981 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.521185 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.521273 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.522236 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.522238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.522238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:38.522244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.522259 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:38.522263 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.522334 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.522432 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.522476 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:19:38.522780 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.517720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36995 (local address 127.18.166.193:54813)
0812 05:19:38.517934 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.517969 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.518528 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.518838 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.518841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.518849 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:38.519352 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.519356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.520087 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.520088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.520886 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.520888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.521285 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.521286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.521474 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.522086 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:38.522086 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:38.522115 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:38.522140 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:38.522335 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:38.522342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:38.522345 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.522434 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.522541 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260812 05:19:38.523313 7786 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36995
I20260812 05:19:38.523422 7786 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:38.523610 7786 heartbeater.cc:511] Master 127.18.166.254:36995 requested a full tablet report, sending...
I20260812 05:19:38.524133 7586 ts_manager.cc:194] Registered new tserver with Master: 5a79a09c605440218952bdd3ef9a4883 (127.18.166.193:42295)
I20260812 05:19:38.524789 7586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.193:54813
I20260812 05:19:38.526779 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7652
I20260812 05:19:38.526907 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-0/wal/instance
I20260812 05:19:38.527132 19099 external_mini_cluster.cc:1428] Setting key 3b5d370b50f2e1a57804e7a42b0f475c
I20260812 05:19:38.528183 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.194:0
--local_ip_for_outbound_sockets=127.18.166.194
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36995
--builtin_ntp_servers=127.18.166.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:38.611356 7789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:38.611604 7789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:38.611644 7789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:38.612890 7789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:38.612951 7789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:38.612999 7789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.194
I20260812 05:19:38.614507 7789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.166.194
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.194
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:38.614773 7789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:38.615078 7789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:38.620122 7794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.620119 7798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.620190 7795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:38.620391 7789 server_base.cc:1034] running on GCE node
I20260812 05:19:38.620628 7789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:38.620877 7789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:38.622020 7789 hybrid_clock.cc:648] HybridClock initialized: now 1786511978621994 us; error 41 us; skew 500 ppm
I20260812 05:19:38.623335 7789 webserver.cc:460] Webserver started at http://127.18.166.194:35017/ using document root <none> and password file <none>
I20260812 05:19:38.623602 7789 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:38.623648 7789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:38.623827 7789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:38.624938 7789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data/instance:
uuid: "4b5a253f3f0a4d4da85bfda3680bd19b"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "0a770fe08e2ee56de47b866c8d82e7cd"
server_key_iv: "370ecca6172301417015d7b90726ec7a"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.625311 7789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance:
uuid: "4b5a253f3f0a4d4da85bfda3680bd19b"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "0a770fe08e2ee56de47b866c8d82e7cd"
server_key_iv: "370ecca6172301417015d7b90726ec7a"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.626820 7789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:38.627611 7804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.627816 7789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:38.627916 7789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal
uuid: "4b5a253f3f0a4d4da85bfda3680bd19b"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "0a770fe08e2ee56de47b866c8d82e7cd"
server_key_iv: "370ecca6172301417015d7b90726ec7a"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.628003 7789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:38.637990 7789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:38.638290 7789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:38.638442 7789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:38.638687 7789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:38.639015 7789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:38.639072 7789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.639125 7789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:38.639153 7789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.645295 7789 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.194:36345
I20260812 05:19:38.645344 7919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.194:36345 every 8 connection(s)
I20260812 05:19:38.645748 7789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/data/info.pb
I20260812 05:19:38.651051 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.646462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:40465 (local address 127.18.166.254:36995)
0812 05:19:38.646513 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.646516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.646743 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:19:38.647294 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.647295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.647308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.647364 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.648072 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.648580 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.649593 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.649796 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.649876 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.650761 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.650763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.650763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:38.650769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.650785 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:38.650789 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.650842 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.650944 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.650971 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:19:38.651275 7922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.646430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36995 (local address 127.18.166.194:40465)
0812 05:19:38.646610 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.646648 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.647179 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.647430 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.647434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.647444 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:38.647971 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.647976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.648691 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.648692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.649498 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.649500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.649889 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.649890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.650027 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.650609 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:38.650610 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:38.650645 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:38.650668 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:38.650850 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:38.650855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:38.650858 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.650945 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.651048 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260812 05:19:38.651778 7920 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36995
I20260812 05:19:38.651861 7920 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:38.652004 7920 heartbeater.cc:511] Master 127.18.166.254:36995 requested a full tablet report, sending...
I20260812 05:19:38.652352 7586 ts_manager.cc:194] Registered new tserver with Master: 4b5a253f3f0a4d4da85bfda3680bd19b (127.18.166.194:36345)
I20260812 05:19:38.652726 7586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.194:40465
I20260812 05:19:38.653395 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7789
I20260812 05:19:38.653549 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal/instance
I20260812 05:19:38.653703 19099 external_mini_cluster.cc:1428] Setting key 205d25caa404cf47ce51ac46a7a8cde7
I20260812 05:19:38.654565 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.166.195:0
--local_ip_for_outbound_sockets=127.18.166.195
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36995
--builtin_ntp_servers=127.18.166.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:19:38.736255 7923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:38.736510 7923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:38.736541 7923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:38.737739 7923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:38.737780 7923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:38.737807 7923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.166.195
I20260812 05:19:38.739133 7923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.166.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.166.195
--webserver_port=0
--tserver_master_addrs=127.18.166.254:36995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.166.195
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:38.739326 7923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:38.739584 7923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:38.744488 7928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.744516 7929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:38.744625 7931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:38.744827 7923 server_base.cc:1034] running on GCE node
I20260812 05:19:38.745076 7923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:38.745287 7923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:38.746522 7923 hybrid_clock.cc:648] HybridClock initialized: now 1786511978746508 us; error 29 us; skew 500 ppm
I20260812 05:19:38.747798 7923 webserver.cc:460] Webserver started at http://127.18.166.195:43461/ using document root <none> and password file <none>
I20260812 05:19:38.748059 7923 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:38.748107 7923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:38.748260 7923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:38.749373 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data/instance:
uuid: "1019dd6cb8444cc48260b18521906bd3"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "236bbdd85c6a70983de987a41e600f15"
server_key_iv: "cabec536a0f44143223fbe7d39a41144"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.749802 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance:
uuid: "1019dd6cb8444cc48260b18521906bd3"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "236bbdd85c6a70983de987a41e600f15"
server_key_iv: "cabec536a0f44143223fbe7d39a41144"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.751154 7923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:19:38.751869 7938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.752048 7923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:38.752148 7923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal
uuid: "1019dd6cb8444cc48260b18521906bd3"
format_stamp: "Formatted at 2026-08-12 05:19:38 on dist-test-slave-srgv"
server_key: "236bbdd85c6a70983de987a41e600f15"
server_key_iv: "cabec536a0f44143223fbe7d39a41144"
server_key_version: "encryptionkey@0"
I20260812 05:19:38.752239 7923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:38.764559 7923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:38.764901 7923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:38.765059 7923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:38.765311 7923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:19:38.765702 7923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:19:38.765760 7923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.765795 7923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:19:38.765826 7923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:38.771616 7923 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.195:43415
I20260812 05:19:38.771690 8053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.195:43415 every 8 connection(s)
I20260812 05:19:38.772035 7923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/data/info.pb
I20260812 05:19:38.777123 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.772823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.195:36087 (local address 127.18.166.254:36995)
0812 05:19:38.772879 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.772882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.773153 (+ 271us) server_negotiation.cc:407] Connection header received
0812 05:19:38.773704 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.773705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.773715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.773769 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.774424 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.774912 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.775849 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.776032 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.776081 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.776883 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.776885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.776885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:38.776890 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.776904 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:38.776909 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.776952 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.777037 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.777060 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:19:38.777347 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.772815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36995 (local address 127.18.166.195:36087)
0812 05:19:38.773025 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.773059 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.773615 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.773820 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.773823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.773831 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:19:38.774349 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.774353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.774994 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.774995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.775773 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.775775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.776107 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.776108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.776215 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.776747 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:19:38.776747 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:19:38.776777 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:19:38.776799 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:19:38.776963 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:19:38.776968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:19:38.776971 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.777023 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.777101 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260812 05:19:38.777956 8054 heartbeater.cc:347] Connected to a master server at 127.18.166.254:36995
I20260812 05:19:38.778067 8054 heartbeater.cc:464] Registering TS with master...
I20260812 05:19:38.778285 8054 heartbeater.cc:511] Master 127.18.166.254:36995 requested a full tablet report, sending...
I20260812 05:19:38.778695 7586 ts_manager.cc:194] Registered new tserver with Master: 1019dd6cb8444cc48260b18521906bd3 (127.18.166.195:43415)
I20260812 05:19:38.779039 7586 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.166.195:36087
I20260812 05:19:38.779757 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 7923
I20260812 05:19:38.779873 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal/instance
I20260812 05:19:38.780045 19099 external_mini_cluster.cc:1428] Setting key 094197f276405ab217c3ad8e344a253f
I20260812 05:19:38.780921 19099 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:19:38.785167 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.782390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.18.166.254:36995)
0812 05:19:38.782444 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.782446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.782563 (+ 117us) server_negotiation.cc:407] Connection header received
0812 05:19:38.782603 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.782604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.782612 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.782668 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:38.782791 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.783248 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.783938 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.784123 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.784194 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.784850 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:38.784851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:38.784851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:38.784855 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:38.784869 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:38.784873 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:38.784914 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.784964 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.784984 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:19:38.786981 7586 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:19:38.788527 7586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:19:38.795058 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.790927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51960 (local address 127.18.166.195:43415)
0812 05:19:38.791108 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.791121 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.791133 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:19:38.791241 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.791243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.791259 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.791576 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.791642 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.792490 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.793908 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.794631 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.794711 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.794803 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.794924 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.794964 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260812 05:19:38.795071 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.790841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43415 (local address 127.0.0.1:51960)
0812 05:19:38.791045 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.791085 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.791155 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.791397 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.791400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.791403 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.791481 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.791483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.792586 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.792587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.793810 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.793813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.794726 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.794728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.794820 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.794836 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.794910 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.794986 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":36,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260812 05:19:38.795295 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.791462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49354 (local address 127.18.166.193:42295)
0812 05:19:38.791659 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.791664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.791674 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:38.791720 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.791722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.791750 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.791957 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.792024 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.793054 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.794200 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.794707 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.795038 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.795137 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.795162 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.795207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260812 05:19:38.795351 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.791317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:42295 (local address 127.0.0.1:49354)
0812 05:19:38.791445 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.791454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.791494 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.791848 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.791849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.791851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.791885 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.791886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.793141 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.793142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.794112 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.794114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.794814 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.794815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.794865 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.794868 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.794899 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.794986 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260812 05:19:38.795693 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.790841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.194:36345 (local address 127.0.0.1:57394)
0812 05:19:38.791296 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.791305 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.791354 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.791628 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.791629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.791631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.791674 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.791676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.792989 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.792990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.794094 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.794096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.795434 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.795434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.795499 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.795502 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.795584 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.795623 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":27,"thread_start_us":86,"threads_started":1}
I20260812 05:19:38.795734 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.791049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.18.166.194:36345)
0812 05:19:38.791219 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.791222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.791415 (+ 193us) server_negotiation.cc:407] Connection header received
0812 05:19:38.791466 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.791469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.791485 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.791557 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.791753 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.792895 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.794194 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.794877 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.794945 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.795020 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.795601 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.795643 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260812 05:19:38.796068 7987 tablet_service.cc:1467] Processing CreateTablet for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 (DEFAULT_TABLE table=test-table [id=a1310f6280b640d9acb858a8ecd3154e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:38.796479 7987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.796391 7852 tablet_service.cc:1467] Processing CreateTablet for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 (DEFAULT_TABLE table=test-table [id=a1310f6280b640d9acb858a8ecd3154e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:38.796491 7715 tablet_service.cc:1467] Processing CreateTablet for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 (DEFAULT_TABLE table=test-table [id=a1310f6280b640d9acb858a8ecd3154e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:19:38.796716 7852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.796877 7715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:38.799868 8068 tablet_bootstrap.cc:492] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883: Bootstrap starting.
I20260812 05:19:38.800283 8070 tablet_bootstrap.cc:492] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Bootstrap starting.
I20260812 05:19:38.800308 8069 tablet_bootstrap.cc:492] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Bootstrap starting.
I20260812 05:19:38.800873 8069 tablet_bootstrap.cc:654] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.800869 8068 tablet_bootstrap.cc:654] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.800891 8070 tablet_bootstrap.cc:654] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:38.801198 8069 log.cc:824] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.801245 8070 log.cc:824] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.801287 8068 log.cc:824] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:38.801990 8069 tablet_bootstrap.cc:492] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: No bootstrap required, opened a new log
I20260812 05:19:38.802048 8069 ts_tablet_manager.cc:1397] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:19:38.802290 8068 tablet_bootstrap.cc:492] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883: No bootstrap required, opened a new log
I20260812 05:19:38.802439 8068 ts_tablet_manager.cc:1397] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 05:19:38.804018 8069 raft_consensus.cc:348] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.804175 8069 raft_consensus.cc:374] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.804199 8069 raft_consensus.cc:729] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5a253f3f0a4d4da85bfda3680bd19b, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.804327 8069 consensus_queue.cc:260] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.804309 8068 raft_consensus.cc:348] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.804484 8068 raft_consensus.cc:374] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.804534 8068 raft_consensus.cc:729] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a79a09c605440218952bdd3ef9a4883, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.804605 8069 ts_tablet_manager.cc:1428] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:19:38.804721 7920 heartbeater.cc:503] Master 127.18.166.254:36995 was elected leader, sending a full tablet report...
I20260812 05:19:38.804682 8068 consensus_queue.cc:260] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.804904 8068 ts_tablet_manager.cc:1428] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:19:38.804981 7786 heartbeater.cc:503] Master 127.18.166.254:36995 was elected leader, sending a full tablet report...
I20260812 05:19:38.805482 8070 tablet_bootstrap.cc:492] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: No bootstrap required, opened a new log
I20260812 05:19:38.805555 8070 ts_tablet_manager.cc:1397] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 05:19:38.807675 8070 raft_consensus.cc:348] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.807821 8070 raft_consensus.cc:374] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:38.807869 8070 raft_consensus.cc:729] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1019dd6cb8444cc48260b18521906bd3, State: Initialized, Role: FOLLOWER
I20260812 05:19:38.807976 8070 consensus_queue.cc:260] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.808225 8070 ts_tablet_manager.cc:1428] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 05:19:38.808315 8054 heartbeater.cc:503] Master 127.18.166.254:36995 was elected leader, sending a full tablet report...
I20260812 05:19:38.822762 8074 raft_consensus.cc:482] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:19:38.822919 8074 raft_consensus.cc:504] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.823334 8074 leader_election.cc:288] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1019dd6cb8444cc48260b18521906bd3 (127.18.166.195:43415), 5a79a09c605440218952bdd3ef9a4883 (127.18.166.193:42295)
I20260812 05:19:38.826144 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.823605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:46557 (local address 127.18.166.193:42295)
0812 05:19:38.823653 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.823655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.823665 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:19:38.823694 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.823695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.823707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.823784 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.823965 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.824449 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.825397 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.825794 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.825886 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.825894 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.826034 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.826073 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:19:38.826144 7922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.823517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:42295 (local address 127.18.166.194:46557)
0812 05:19:38.823567 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.823580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.823625 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.823803 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.823804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.823805 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.823883 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.823885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.824541 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.824542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.825294 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.825296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.825883 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.825884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.825959 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.825962 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.826029 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.826073 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 05:19:38.826486 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.823666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.166.194:33195 (local address 127.18.166.195:43415)
0812 05:19:38.823722 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.823724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.823854 (+ 130us) server_negotiation.cc:407] Connection header received
0812 05:19:38.823936 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.823937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.823945 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.824018 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:19:38.824191 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.824651 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.825638 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.826065 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.826151 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.826158 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.826364 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.826413 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:19:38.826514 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4a9d146d464a92b0e7b14a4ebcf9a2" candidate_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a79a09c605440218952bdd3ef9a4883" is_pre_election: true
I20260812 05:19:38.826670 7736 raft_consensus.cc:2454] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b5a253f3f0a4d4da85bfda3680bd19b in term 0.
I20260812 05:19:38.826745 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.823519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.195:43415 (local address 127.18.166.194:33195)
0812 05:19:38.823775 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.823787 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:19:38.823868 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:19:38.824027 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:19:38.824027 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:19:38.824029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:19:38.824122 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.824123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.824746 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.824747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.825550 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:19:38.825552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.826197 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.826198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:19:38.826278 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.826281 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:19:38.826578 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 05:19:38.826632 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":55,"thread_start_us":143,"threads_started":1}
I20260812 05:19:38.826965 7807 leader_election.cc:302] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b5a253f3f0a4d4da85bfda3680bd19b, 5a79a09c605440218952bdd3ef9a4883; no voters:
I20260812 05:19:38.827093 8074 raft_consensus.cc:2781] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:19:38.827159 8074 raft_consensus.cc:482] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:19:38.827090 8007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4a9d146d464a92b0e7b14a4ebcf9a2" candidate_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019dd6cb8444cc48260b18521906bd3" is_pre_election: true
I20260812 05:19:38.827198 8074 raft_consensus.cc:3037] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.827237 8007 raft_consensus.cc:2454] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b5a253f3f0a4d4da85bfda3680bd19b in term 0.
I20260812 05:19:38.827955 8074 raft_consensus.cc:504] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.828102 8074 leader_election.cc:288] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [CANDIDATE]: Term 1 election: Requested vote from peers 1019dd6cb8444cc48260b18521906bd3 (127.18.166.195:43415), 5a79a09c605440218952bdd3ef9a4883 (127.18.166.193:42295)
I20260812 05:19:38.828279 8007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4a9d146d464a92b0e7b14a4ebcf9a2" candidate_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019dd6cb8444cc48260b18521906bd3"
I20260812 05:19:38.828346 8007 raft_consensus.cc:3037] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.829044 8007 raft_consensus.cc:2454] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b5a253f3f0a4d4da85bfda3680bd19b in term 1.
I20260812 05:19:38.829167 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4a9d146d464a92b0e7b14a4ebcf9a2" candidate_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a79a09c605440218952bdd3ef9a4883"
I20260812 05:19:38.829219 7808 leader_election.cc:302] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1019dd6cb8444cc48260b18521906bd3, 4b5a253f3f0a4d4da85bfda3680bd19b; no voters:
I20260812 05:19:38.829319 8074 raft_consensus.cc:2781] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:38.829277 7736 raft_consensus.cc:3037] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:38.829524 8074 raft_consensus.cc:686] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 1 LEADER]: Becoming Leader. State: Replica: 4b5a253f3f0a4d4da85bfda3680bd19b, State: Running, Role: LEADER
I20260812 05:19:38.829645 8074 consensus_queue.cc:237] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } }
I20260812 05:19:38.830276 7736 raft_consensus.cc:2454] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b5a253f3f0a4d4da85bfda3680bd19b in term 1.
I20260812 05:19:38.830546 7585 catalog_manager.cc:5495] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b5a253f3f0a4d4da85bfda3680bd19b (127.18.166.194). New cstate: current_term: 1 leader_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b5a253f3f0a4d4da85bfda3680bd19b" member_type: VOTER last_known_addr { host: "127.18.166.194" port: 36345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 } health_report { overall_health: UNKNOWN } } }
I20260812 05:19:38.840431 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.838316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.18.166.194:36345)
0812 05:19:38.838363 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:38.838365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:38.838373 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:19:38.838423 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:38.838424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:38.838440 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:38.838501 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:19:38.838595 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.839046 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.839783 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:38.839981 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:38.840037 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:38.840156 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:19:38.840249 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:19:38.840305 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:38.840336 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 05:19:38.840351 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:38.842294 8007 raft_consensus.cc:1261] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 1 FOLLOWER]: Refusing update from remote peer 4b5a253f3f0a4d4da85bfda3680bd19b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:38.842295 7736 raft_consensus.cc:1261] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 5a79a09c605440218952bdd3ef9a4883 [term 1 FOLLOWER]: Refusing update from remote peer 4b5a253f3f0a4d4da85bfda3680bd19b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:19:38.842657 8074 consensus_queue.cc:1035] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1019dd6cb8444cc48260b18521906bd3" member_type: VOTER last_known_addr { host: "127.18.166.195" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:38.842805 8079 consensus_queue.cc:1035] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a79a09c605440218952bdd3ef9a4883" member_type: VOTER last_known_addr { host: "127.18.166.193" port: 42295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:19:38.845335 8082 mvcc.cc:205] Tried to move back new op lower bound from 7317553065335349248 to 7317553065286828032. Current Snapshot: MvccSnapshot[applied={T|T < 7317553065335349248}]
I20260812 05:19:38.845557 8084 mvcc.cc:205] Tried to move back new op lower bound from 7317553065335349248 to 7317553065286828032. Current Snapshot: MvccSnapshot[applied={T|T < 7317553065335349248}]
I20260812 05:19:38.846063 8083 mvcc.cc:205] Tried to move back new op lower bound from 7317553065335349248 to 7317553065286828032. Current Snapshot: MvccSnapshot[applied={T|T < 7317553065335349248 or (T in {7317553065335349248})}]
I20260812 05:19:38.848135 7586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:38.848242 7586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:19:38.849009 7586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebd24e1334ce4627866ba61029b8225b: Sending DeleteTablet for 3 replicas of tablet cc4a9d146d464a92b0e7b14a4ebcf9a2
I20260812 05:19:38.849360 7852 tablet_service.cc:1514] Processing DeleteTablet for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:38 UTC) from {username='slave'} at 127.0.0.1:57394
I20260812 05:19:38.849372 7987 tablet_service.cc:1514] Processing DeleteTablet for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:38 UTC) from {username='slave'} at 127.0.0.1:51960
I20260812 05:19:38.849675 7715 tablet_service.cc:1514] Processing DeleteTablet for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:19:38 UTC) from {username='slave'} at 127.0.0.1:49354
I20260812 05:19:38.849792 8094 tablet_replica.cc:330] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: stopping tablet replica
I20260812 05:19:38.849895 8094 raft_consensus.cc:2229] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:19:38.849997 8094 raft_consensus.cc:2258] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:38.850013 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7652
I20260812 05:19:38.850172 8093 tablet_replica.cc:330] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: stopping tablet replica
I20260812 05:19:38.850257 8093 raft_consensus.cc:2229] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:19:38.852746 8093 raft_consensus.cc:2258] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:19:38.852767 8094 ts_tablet_manager.cc:1905] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:38.853206 8093 ts_tablet_manager.cc:1905] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:19:38.854635 8094 ts_tablet_manager.cc:1918] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:38.854813 8094 log.cc:1196] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-2/wal/wals/cc4a9d146d464a92b0e7b14a4ebcf9a2
I20260812 05:19:38.854950 8093 ts_tablet_manager.cc:1918] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:19:38.855026 8093 log.cc:1196] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Deleting WAL directory at /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786511731160932-19099-0/minicluster-data/ts-1/wal/wals/cc4a9d146d464a92b0e7b14a4ebcf9a2
I20260812 05:19:38.855265 8094 ts_tablet_manager.cc:1939] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 1019dd6cb8444cc48260b18521906bd3: Deleting consensus metadata
I20260812 05:19:38.855288 8093 ts_tablet_manager.cc:1939] T cc4a9d146d464a92b0e7b14a4ebcf9a2 P 4b5a253f3f0a4d4da85bfda3680bd19b: Deleting consensus metadata
I20260812 05:19:38.855672 7573 catalog_manager.cc:4841] TS 1019dd6cb8444cc48260b18521906bd3 (127.18.166.195:43415): tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 (table test-table [id=a1310f6280b640d9acb858a8ecd3154e]) successfully deleted
I20260812 05:19:38.855669 7570 catalog_manager.cc:4841] TS 4b5a253f3f0a4d4da85bfda3680bd19b (127.18.166.194:36345): tablet cc4a9d146d464a92b0e7b14a4ebcf9a2 (table test-table [id=a1310f6280b640d9acb858a8ecd3154e]) successfully deleted
W20260812 05:19:38.859431 7572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.166.193:42295 (error 108)
I20260812 05:19:38.859926 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:38.859805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.193:42295 (local address 127.0.0.1:49358)
0812 05:19:38.859848 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 05:19:38.859889 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.193:42295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 05:19:38.860060 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7789
W20260812 05:19:38.860059 7572 catalog_manager.cc:4584] TS 5a79a09c605440218952bdd3ef9a4883 (127.18.166.193:42295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc4a9d146d464a92b0e7b14a4ebcf9a2: Network error: Client connection negotiation failed: client connection to 127.18.166.193:42295: connect: Connection refused (error 111)
I20260812 05:19:38.866607 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7923
I20260812 05:19:38.872680 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 7554
2026-08-12T05:19:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (666 ms)
[----------] 6 tests from EncryptionPolicyTest (9015 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:38 dist-test-slave-srgv krb5kdc[8100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:38 dist-test-slave-srgv krb5kdc[8100](info): set up 2 sockets
Aug 12 05:19:38 dist-test-slave-srgv krb5kdc[8100](info): commencing operation
krb5kdc: starting...
W20260812 05:19:39.938683 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:40 dist-test-slave-srgv krb5kdc[8100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:40.062076 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:40.144603 8113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:40.144815 8113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:40.144840 8113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:40.146039 8113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:40.146085 8113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:40.146099 8113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:40.146112 8113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:40.146124 8113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:40.147679 8113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:40.147883 8113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:40.148053 8113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:40.152964 8118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:40.153014 8119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:40.153117 8113 server_base.cc:1034] running on GCE node
W20260812 05:19:40.152981 8122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:40.153553 8113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:40.153869 8113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:40.155016 8113 hybrid_clock.cc:648] HybridClock initialized: now 1786511980155003 us; error 29 us; skew 500 ppm
Aug 12 05:19:40 dist-test-slave-srgv krb5kdc[8100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511980, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:40.160641 8113 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:40.161127 8113 webserver.cc:460] Webserver started at http://127.0.0.1:42969/ using document root <none> and password file <none>
I20260812 05:19:40.161326 8113 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:40.161366 8113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:40.161523 8113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:40.162575 8113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "430f5cde836d42b4a5f9c4c952284650"
format_stamp: "Formatted at 2026-08-12 05:19:40 on dist-test-slave-srgv"
server_key: "21820668606f2047354d0ddd79d9ce0b"
server_key_iv: "9210c5dc9265a0db5a0cd215616a629e"
server_key_version: "encryptionkey@0"
I20260812 05:19:40.162883 8113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "430f5cde836d42b4a5f9c4c952284650"
format_stamp: "Formatted at 2026-08-12 05:19:40 on dist-test-slave-srgv"
server_key: "21820668606f2047354d0ddd79d9ce0b"
server_key_iv: "9210c5dc9265a0db5a0cd215616a629e"
server_key_version: "encryptionkey@0"
I20260812 05:19:40.164258 8113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:19:40.164992 8129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:40.165206 8113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:40.165328 8113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "430f5cde836d42b4a5f9c4c952284650"
format_stamp: "Formatted at 2026-08-12 05:19:40 on dist-test-slave-srgv"
server_key: "21820668606f2047354d0ddd79d9ce0b"
server_key_iv: "9210c5dc9265a0db5a0cd215616a629e"
server_key_version: "encryptionkey@0"
I20260812 05:19:40.165409 8113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:40.194731 8113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:40.197062 8113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:40.197248 8113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:40.201606 8113 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44261
I20260812 05:19:40.201647 8183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44261 every 8 connection(s)
I20260812 05:19:40.202049 8113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:40.202994 8184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:40.205444 8184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650: Bootstrap starting.
I20260812 05:19:40.206014 8184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:40.206262 8184 log.cc:824] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:40.206959 8184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650: No bootstrap required, opened a new log
I20260812 05:19:40.207834 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8113
I20260812 05:19:40.207966 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:40.208194 19099 external_mini_cluster.cc:1428] Setting key 0ba82c424a450a6d1f6727f753f3e421
I20260812 05:19:40.209210 8184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430f5cde836d42b4a5f9c4c952284650" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44261 } }
I20260812 05:19:40.209390 8184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:40.209484 8184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f5cde836d42b4a5f9c4c952284650, State: Initialized, Role: FOLLOWER
I20260812 05:19:40.209633 8184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430f5cde836d42b4a5f9c4c952284650" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44261 } }
I20260812 05:19:40.209703 8184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:40.209725 8184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:40.209800 8184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:40.210953 8184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430f5cde836d42b4a5f9c4c952284650" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44261 } }
I20260812 05:19:40.211068 8184 leader_election.cc:302] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 430f5cde836d42b4a5f9c4c952284650; no voters:
I20260812 05:19:40.211236 8184 leader_election.cc:288] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:40.211304 8189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:40.211467 8189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [term 1 LEADER]: Becoming Leader. State: Replica: 430f5cde836d42b4a5f9c4c952284650, State: Running, Role: LEADER
I20260812 05:19:40.211496 8184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:40.211558 8189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430f5cde836d42b4a5f9c4c952284650" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44261 } }
I20260812 05:19:40.211899 8190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "430f5cde836d42b4a5f9c4c952284650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430f5cde836d42b4a5f9c4c952284650" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44261 } } }
I20260812 05:19:40.212006 8190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:40.212291 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 430f5cde836d42b4a5f9c4c952284650. Latest consensus state: current_term: 1 leader_uuid: "430f5cde836d42b4a5f9c4c952284650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430f5cde836d42b4a5f9c4c952284650" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44261 } } }
I20260812 05:19:40.212385 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:40.212505 8197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:40.213047 8197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:40.215212 8197 catalog_manager.cc:1347] Generated new cluster ID: 0f0c6356429a4dd88f14c5d2fe3fda5d
I20260812 05:19:40.215267 8197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:19:40 dist-test-slave-srgv krb5kdc[8100](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:19:40.220569 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:40.208812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59845 (local address 127.0.0.1:44261)
0812 05:19:40.209020 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:40.209031 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:40.209071 (+ 40us) server_negotiation.cc:407] Connection header received
0812 05:19:40.209204 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:40.209211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:40.209280 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:40.209473 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:40.210285 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:40.211758 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:40.213026 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:40.213717 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:40.213837 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:40.216617 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:40.216621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:40.216621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:40.216633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:40.218465 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:40.218935 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:40.218944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:40.218945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:40.219062 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:40.219328 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:40.219330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:40.219330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:40.219596 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:40.220245 (+ 649us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:40.220319 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 05:19:40.220406 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260812 05:19:40.221756 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:40.229833 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:40.223224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37805 (local address 127.0.0.1:44261)
0812 05:19:40.223265 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:40.223268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:40.223384 (+ 116us) server_negotiation.cc:407] Connection header received
0812 05:19:40.223460 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:40.223461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:40.223471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:40.223528 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:40.224095 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:40.224628 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:40.225336 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:40.225650 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:40.225740 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:40.227034 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:40.227036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:40.227036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:40.227044 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:40.228570 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:40.228949 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:40.228951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:40.228951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:40.229013 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:40.229243 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:40.229244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:40.229244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:40.229363 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:40.229513 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:40.229646 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 05:19:40.229711 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:19:40.237653 8197 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:40.238235 8197 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:40.248103 8197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 430f5cde836d42b4a5f9c4c952284650: Generated new TSK 0
I20260812 05:19:40.248353 8197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:40.254032 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8113
W20260812 05:19:40.260120 8098 connection.cc:535] client connection to 127.0.0.1:44261 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44261): Connection reset by peer (error 104)
2026-08-12T05:19:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:40 dist-test-slave-srgv krb5kdc[8216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:40 dist-test-slave-srgv krb5kdc[8216](info): set up 2 sockets
Aug 12 05:19:40 dist-test-slave-srgv krb5kdc[8216](info): commencing operation
krb5kdc: starting...
W20260812 05:19:41.322103 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:41 dist-test-slave-srgv krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:41.448992 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:41.534377 8230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:41.534618 8230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:41.534657 8230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:41.535784 8230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:41.535851 8230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:41.535879 8230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:41.535910 8230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:41.535940 8230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:41.537469 8230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40461
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:41.537767 8230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:41.537998 8230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:41.543123 8236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:41.543121 8238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:41.543355 8235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:41.544121 8230 server_base.cc:1034] running on GCE node
I20260812 05:19:41.544415 8230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:41.544741 8230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:41.545930 8230 hybrid_clock.cc:648] HybridClock initialized: now 1786511981545899 us; error 43 us; skew 500 ppm
Aug 12 05:19:41 dist-test-slave-srgv krb5kdc[8216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511981, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:41.549525 8230 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:41.550410 8230 webserver.cc:460] Webserver started at http://127.0.0.1:36935/ using document root <none> and password file <none>
I20260812 05:19:41.550724 8230 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:41.550784 8230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:41.550889 8230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:41.551927 8230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "62a6dfa44f6a4ee296678107b5895af6"
format_stamp: "Formatted at 2026-08-12 05:19:41 on dist-test-slave-srgv"
server_key: "9d29978682a4e2920271bef9d5e546b3"
server_key_iv: "5c24fd4c21b7b71acfb8002f3cd739cc"
server_key_version: "encryptionkey@0"
I20260812 05:19:41.552222 8230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "62a6dfa44f6a4ee296678107b5895af6"
format_stamp: "Formatted at 2026-08-12 05:19:41 on dist-test-slave-srgv"
server_key: "9d29978682a4e2920271bef9d5e546b3"
server_key_iv: "5c24fd4c21b7b71acfb8002f3cd739cc"
server_key_version: "encryptionkey@0"
I20260812 05:19:41.553515 8230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:41.554874 8245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:41.555049 8230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:41.555150 8230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "62a6dfa44f6a4ee296678107b5895af6"
format_stamp: "Formatted at 2026-08-12 05:19:41 on dist-test-slave-srgv"
server_key: "9d29978682a4e2920271bef9d5e546b3"
server_key_iv: "5c24fd4c21b7b71acfb8002f3cd739cc"
server_key_version: "encryptionkey@0"
I20260812 05:19:41.555238 8230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:41.593372 8230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:41.595593 8230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:41.595781 8230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:41.600095 8230 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40461
I20260812 05:19:41.600137 8299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40461 every 8 connection(s)
I20260812 05:19:41.600668 8230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:41.601398 8300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:41.603706 8300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6: Bootstrap starting.
I20260812 05:19:41.604249 8300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:41.604519 8300 log.cc:824] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:41.604614 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8230
I20260812 05:19:41.604784 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:41.605038 19099 external_mini_cluster.cc:1428] Setting key b703bdaca88ec8b8285b94d3ffcf6c99
I20260812 05:19:41.605396 8300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6: No bootstrap required, opened a new log
I20260812 05:19:41.607218 8300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a6dfa44f6a4ee296678107b5895af6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 05:19:41.607388 8300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:41.607427 8300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a6dfa44f6a4ee296678107b5895af6, State: Initialized, Role: FOLLOWER
I20260812 05:19:41.607565 8300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a6dfa44f6a4ee296678107b5895af6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 05:19:41.607667 8300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:41.607733 8300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:41.607782 8300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:41.608341 8300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a6dfa44f6a4ee296678107b5895af6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 05:19:41.608426 8300 leader_election.cc:302] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62a6dfa44f6a4ee296678107b5895af6; no voters:
I20260812 05:19:41.608614 8300 leader_election.cc:288] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:41.608635 8305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:41.608749 8305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [term 1 LEADER]: Becoming Leader. State: Replica: 62a6dfa44f6a4ee296678107b5895af6, State: Running, Role: LEADER
I20260812 05:19:41.608829 8305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a6dfa44f6a4ee296678107b5895af6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } }
I20260812 05:19:41.608891 8300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:41.609169 8307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62a6dfa44f6a4ee296678107b5895af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a6dfa44f6a4ee296678107b5895af6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } } }
I20260812 05:19:41.609459 8307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:41.609330 8306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62a6dfa44f6a4ee296678107b5895af6. Latest consensus state: current_term: 1 leader_uuid: "62a6dfa44f6a4ee296678107b5895af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62a6dfa44f6a4ee296678107b5895af6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40461 } } }
I20260812 05:19:41.609725 8306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6 [sys.catalog]: This master's current role is: LEADER
W20260812 05:19:41.610704 8322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:19:41.610749 8322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:19:41.610818 8315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:41.610981 8315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:41.612581 8315 catalog_manager.cc:1347] Generated new cluster ID: f094b10caa8b41cfb1ac0d902831302a
I20260812 05:19:41.612649 8315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:41.612731 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:41.605668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38245 (local address 127.0.0.1:40461)
0812 05:19:41.605861 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:41.605874 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:41.605902 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:19:41.607205 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:41.607212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:41.607259 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:41.607376 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:41.607634 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:41.609188 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:41.610477 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:41.611076 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:41.611180 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:41.611961 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:41.611964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:41.611965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:41.612214 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:41.612256 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:41.612275 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:41.612374 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:41.612457 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:19:41.612485 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260812 05:19:41.613529 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:41.619102 8315 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:41.619792 8315 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:41.620996 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:41.614966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55369 (local address 127.0.0.1:40461)
0812 05:19:41.614996 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:41.614997 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:41.617612 (+ 2615us) server_negotiation.cc:407] Connection header received
0812 05:19:41.617703 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:41.617704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:41.617715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:41.617742 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:41.617915 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:41.618594 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:41.619622 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:41.619902 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:41.619948 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:41.620732 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:41.620733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:41.620733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:41.620737 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:41.620751 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:41.620756 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:41.620784 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:41.620894 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 05:19:41.620906 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 05:19:41.630576 8315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62a6dfa44f6a4ee296678107b5895af6: Generated new TSK 0
I20260812 05:19:41.630805 8315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:41.633028 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8230
W20260812 05:19:41.639562 8214 connection.cc:535] client connection to 127.0.0.1:40461 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40461): Connection reset by peer (error 104)
2026-08-12T05:19:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:41 dist-test-slave-srgv krb5kdc[8332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:41 dist-test-slave-srgv krb5kdc[8332](info): set up 2 sockets
Aug 12 05:19:41 dist-test-slave-srgv krb5kdc[8332](info): commencing operation
krb5kdc: starting...
W20260812 05:19:42.702762 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:42 dist-test-slave-srgv krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:42.828128 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:42.909703 8346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:42.909969 8346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:42.910009 8346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:42.911191 8346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:42.911279 8346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:42.911309 8346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:42.911340 8346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:42.911362 8346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:42.912864 8346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:42.913160 8346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:42.913439 8346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:42.918756 8360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:42.918697 8355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:42.918727 8346 server_base.cc:1034] running on GCE node
W20260812 05:19:42.918697 8352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:42.919272 8346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:42.919557 8346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:42.920740 8346 hybrid_clock.cc:648] HybridClock initialized: now 1786511982920723 us; error 32 us; skew 500 ppm
Aug 12 05:19:42 dist-test-slave-srgv krb5kdc[8332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:42.923473 8346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:42.923877 8346 webserver.cc:460] Webserver started at http://127.0.0.1:41733/ using document root <none> and password file <none>
I20260812 05:19:42.924072 8346 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:42.924113 8346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:42.924211 8346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:42.925232 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "5e2fb2b305c04da59273ce8b972a5088"
format_stamp: "Formatted at 2026-08-12 05:19:42 on dist-test-slave-srgv"
server_key: "6ae3f8c0066ec2d1072e61e0abd6add0"
server_key_iv: "464dcd5a9904542fc19011f2c6ebb117"
server_key_version: "encryptionkey@0"
I20260812 05:19:42.925616 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "5e2fb2b305c04da59273ce8b972a5088"
format_stamp: "Formatted at 2026-08-12 05:19:42 on dist-test-slave-srgv"
server_key: "6ae3f8c0066ec2d1072e61e0abd6add0"
server_key_iv: "464dcd5a9904542fc19011f2c6ebb117"
server_key_version: "encryptionkey@0"
I20260812 05:19:42.926927 8346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:42.927639 8367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:42.927819 8346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:42.927920 8346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "5e2fb2b305c04da59273ce8b972a5088"
format_stamp: "Formatted at 2026-08-12 05:19:42 on dist-test-slave-srgv"
server_key: "6ae3f8c0066ec2d1072e61e0abd6add0"
server_key_iv: "464dcd5a9904542fc19011f2c6ebb117"
server_key_version: "encryptionkey@0"
I20260812 05:19:42.928004 8346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:42.931402 8346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:42.933779 8346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:42.933939 8346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:42.938225 8346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46051
I20260812 05:19:42.938264 8420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46051 every 8 connection(s)
I20260812 05:19:42.938602 8346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:42.939458 8421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:42.941884 8421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088: Bootstrap starting.
I20260812 05:19:42.942579 8421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:42.942895 8421 log.cc:824] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:42.943279 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8346
I20260812 05:19:42.943428 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:42.943686 19099 external_mini_cluster.cc:1428] Setting key 40c9d2ea2c44e8fb2d044bca81fc87fa
I20260812 05:19:42.943806 8421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088: No bootstrap required, opened a new log
I20260812 05:19:42.946380 8421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fb2b305c04da59273ce8b972a5088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46051 } }
I20260812 05:19:42.946578 8421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:42.946621 8421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2fb2b305c04da59273ce8b972a5088, State: Initialized, Role: FOLLOWER
I20260812 05:19:42.946796 8421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fb2b305c04da59273ce8b972a5088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46051 } }
I20260812 05:19:42.946897 8421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:42.946920 8421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:42.946978 8421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:42.947229 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:42.944323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51691 (local address 127.0.0.1:46051)
0812 05:19:42.944549 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:42.944559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:42.944599 (+ 40us) server_negotiation.cc:407] Connection header received
0812 05:19:42.945415 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:42.945506 (+ 91us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:42.945554 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:42.945718 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:42.946432 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:42.946436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:42.946436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:42.946440 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:42.946489 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:42.946503 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:42.946582 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:42.946721 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:19:42.946784 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260812 05:19:42.947824 8421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fb2b305c04da59273ce8b972a5088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46051 } }
I20260812 05:19:42.947945 8421 leader_election.cc:302] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e2fb2b305c04da59273ce8b972a5088; no voters:
I20260812 05:19:42.948118 8421 leader_election.cc:288] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:42.948148 8427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:42.948300 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:42.948396 8421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:42.949702 8427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2fb2b305c04da59273ce8b972a5088, State: Running, Role: LEADER
I20260812 05:19:42.949844 8427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fb2b305c04da59273ce8b972a5088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46051 } }
I20260812 05:19:42.950413 8428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e2fb2b305c04da59273ce8b972a5088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fb2b305c04da59273ce8b972a5088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46051 } } }
I20260812 05:19:42.950528 8428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:42.950960 8447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:42.951133 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:42.949897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50319 (local address 127.0.0.1:46051)
0812 05:19:42.949954 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:42.949956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:42.950043 (+ 87us) server_negotiation.cc:407] Connection header received
0812 05:19:42.950103 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:42.950103 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:42.950107 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:42.950179 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:42.950861 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:42.950862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:42.950863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:42.950867 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:42.950882 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:42.950887 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:42.950953 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:42.951037 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 05:19:42.951049 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:19:42.951170 8430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e2fb2b305c04da59273ce8b972a5088. Latest consensus state: current_term: 1 leader_uuid: "5e2fb2b305c04da59273ce8b972a5088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2fb2b305c04da59273ce8b972a5088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46051 } } }
I20260812 05:19:42.951421 8430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:42.951756 8447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:42.953241 8447 catalog_manager.cc:1347] Generated new cluster ID: 6a0a9fa8e04c491b86864f468c5fddb9
I20260812 05:19:42.953296 8447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:42.962935 8447 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:42.963125 8447 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:42.974596 8447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e2fb2b305c04da59273ce8b972a5088: Generated new TSK 0
I20260812 05:19:42.974824 8447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:42.980331 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8346
2026-08-12T05:19:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:43 dist-test-slave-srgv krb5kdc[8455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:43 dist-test-slave-srgv krb5kdc[8455](info): set up 2 sockets
Aug 12 05:19:43 dist-test-slave-srgv krb5kdc[8455](info): commencing operation
krb5kdc: starting...
W20260812 05:19:44.046734 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:44 dist-test-slave-srgv krb5kdc[8455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:44.169862 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:44.252302 8468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:44.252589 8468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:44.252645 8468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:44.253860 8468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:44.253944 8468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:44.253970 8468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:44.253998 8468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:44.254029 8468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:44.255517 8468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:44.255820 8468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:44.256060 8468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:44.261265 8474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:44.261315 8476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:44.261291 8473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:44.261644 8468 server_base.cc:1034] running on GCE node
I20260812 05:19:44.261935 8468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:44.262250 8468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:44.263437 8468 hybrid_clock.cc:648] HybridClock initialized: now 1786511984263420 us; error 30 us; skew 500 ppm
Aug 12 05:19:44 dist-test-slave-srgv krb5kdc[8455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:44.266186 8468 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:44.266685 8468 webserver.cc:460] Webserver started at http://127.0.0.1:35745/ using document root <none> and password file <none>
I20260812 05:19:44.266932 8468 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:44.267004 8468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:44.267160 8468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:44.268302 8468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "cb7a97fcb6ef40bdb42d454a95663f9c"
format_stamp: "Formatted at 2026-08-12 05:19:44 on dist-test-slave-srgv"
server_key: "2e815e659923675df223fd432ae650db"
server_key_iv: "59ea3925846d3c01088df8effe2c2282"
server_key_version: "encryptionkey@0"
I20260812 05:19:44.268705 8468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "cb7a97fcb6ef40bdb42d454a95663f9c"
format_stamp: "Formatted at 2026-08-12 05:19:44 on dist-test-slave-srgv"
server_key: "2e815e659923675df223fd432ae650db"
server_key_iv: "59ea3925846d3c01088df8effe2c2282"
server_key_version: "encryptionkey@0"
I20260812 05:19:44.270256 8468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:19:44.271076 8483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:44.271279 8468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:44.271390 8468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "cb7a97fcb6ef40bdb42d454a95663f9c"
format_stamp: "Formatted at 2026-08-12 05:19:44 on dist-test-slave-srgv"
server_key: "2e815e659923675df223fd432ae650db"
server_key_iv: "59ea3925846d3c01088df8effe2c2282"
server_key_version: "encryptionkey@0"
I20260812 05:19:44.271471 8468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:44.316365 8468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:44.318724 8468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:44.318907 8468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:44.323143 8468 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33117
I20260812 05:19:44.323189 8535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33117 every 8 connection(s)
I20260812 05:19:44.323581 8468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:44.324548 8536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:44.325521 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8468
I20260812 05:19:44.325646 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:44.325846 19099 external_mini_cluster.cc:1428] Setting key 04ab744fb3094d77d809d76900cc7af1
I20260812 05:19:44.327661 8536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c: Bootstrap starting.
I20260812 05:19:44.328367 8536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:44.328711 8536 log.cc:824] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:44.329519 8536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c: No bootstrap required, opened a new log
I20260812 05:19:44.331885 8536 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260812 05:19:44.332023 8536 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:44.332046 8536 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb7a97fcb6ef40bdb42d454a95663f9c, State: Initialized, Role: FOLLOWER
I20260812 05:19:44.332197 8536 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260812 05:19:44.332278 8536 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:44.332304 8536 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:44.332376 8536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:44.333062 8536 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260812 05:19:44.333190 8536 leader_election.cc:302] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb7a97fcb6ef40bdb42d454a95663f9c; no voters:
I20260812 05:19:44.333388 8536 leader_election.cc:288] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 12 05:19:44 dist-test-slave-srgv krb5kdc[8455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:19:44.333492 8541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:44.333648 8541 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [term 1 LEADER]: Becoming Leader. State: Replica: cb7a97fcb6ef40bdb42d454a95663f9c, State: Running, Role: LEADER
I20260812 05:19:44.333686 8536 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:44.333736 8541 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260812 05:19:44.334164 8543 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb7a97fcb6ef40bdb42d454a95663f9c. Latest consensus state: current_term: 1 leader_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } } }
I20260812 05:19:44.334167 8542 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7a97fcb6ef40bdb42d454a95663f9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } } }
I20260812 05:19:44.334300 8543 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:44.334306 8542 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:44.335328 8549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:44.336020 8549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:44.337742 8549 catalog_manager.cc:1347] Generated new cluster ID: 69fe91da4eac4be69c5ef229e1ea0bd4
I20260812 05:19:44.337795 8549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:44.338765 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:44.326394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48007 (local address 127.0.0.1:33117)
0812 05:19:44.326592 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:44.326611 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:44.326650 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:19:44.327350 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:44.327359 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:44.327423 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:44.327585 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:44.328229 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:44.329684 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:44.330428 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:44.331059 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:44.331194 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:44.334120 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:44.334124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:44.334125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:44.334139 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:44.336575 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:44.336984 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:44.336988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:44.336988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:44.337122 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:44.337393 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:44.337394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:44.337395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:44.337736 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:44.337876 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:44.338401 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 05:19:44.338540 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260812 05:19:44.339764 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:44.349237 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:44.341260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46701 (local address 127.0.0.1:33117)
0812 05:19:44.341297 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:44.341298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:44.341502 (+ 204us) server_negotiation.cc:407] Connection header received
0812 05:19:44.341612 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:44.341614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:44.341626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:44.341660 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:44.342453 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:44.343199 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:44.344286 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:44.344629 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:44.344676 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:44.346263 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:44.346264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:44.346265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:44.346272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:44.347938 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:44.348357 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:44.348359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:44.348359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:44.348418 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:44.348619 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:44.348620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:44.348620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:44.348754 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:44.348827 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:44.349039 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 05:19:44.349133 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:19:44.353307 8549 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:44.354111 8549 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:44.356490 8549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb7a97fcb6ef40bdb42d454a95663f9c: Generated new TSK 0
I20260812 05:19:44.356709 8549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:44.358920 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8468
2026-08-12T05:19:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:44 dist-test-slave-srgv krb5kdc[8570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:44 dist-test-slave-srgv krb5kdc[8570](info): set up 2 sockets
Aug 12 05:19:44 dist-test-slave-srgv krb5kdc[8570](info): commencing operation
krb5kdc: starting...
W20260812 05:19:45.430603 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:45 dist-test-slave-srgv krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:45.554473 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:45.637117 8584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:45.637359 8584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:45.637385 8584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:45.638850 8584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:45.638924 8584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:45.638964 8584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:45.639001 8584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:45.639039 8584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:45.640733 8584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:45.641008 8584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:45.641251 8584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:45.646426 8590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:45.646509 8589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:45.646569 8584 server_base.cc:1034] running on GCE node
W20260812 05:19:45.646426 8592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:45.646863 8584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:45.647207 8584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:45.648365 8584 hybrid_clock.cc:648] HybridClock initialized: now 1786511985648343 us; error 38 us; skew 500 ppm
Aug 12 05:19:45 dist-test-slave-srgv krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:45.654862 8584 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:45.655388 8584 webserver.cc:460] Webserver started at http://127.0.0.1:42443/ using document root <none> and password file <none>
I20260812 05:19:45.655607 8584 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:45.655649 8584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:45.655743 8584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:45.656811 8584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "f97831c738da4c9c8c7d2a1a2a01cdae"
format_stamp: "Formatted at 2026-08-12 05:19:45 on dist-test-slave-srgv"
server_key: "f82856c34ad45d429feb1c3aa9f3c81c"
server_key_iv: "6aeb9204c8e319b084d4e147c8e370ba"
server_key_version: "encryptionkey@0"
I20260812 05:19:45.657133 8584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "f97831c738da4c9c8c7d2a1a2a01cdae"
format_stamp: "Formatted at 2026-08-12 05:19:45 on dist-test-slave-srgv"
server_key: "f82856c34ad45d429feb1c3aa9f3c81c"
server_key_iv: "6aeb9204c8e319b084d4e147c8e370ba"
server_key_version: "encryptionkey@0"
I20260812 05:19:45.658595 8584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:19:45.659339 8599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:45.659521 8584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:45.659608 8584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "f97831c738da4c9c8c7d2a1a2a01cdae"
format_stamp: "Formatted at 2026-08-12 05:19:45 on dist-test-slave-srgv"
server_key: "f82856c34ad45d429feb1c3aa9f3c81c"
server_key_iv: "6aeb9204c8e319b084d4e147c8e370ba"
server_key_version: "encryptionkey@0"
I20260812 05:19:45.659667 8584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:45.672613 8584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:45.675016 8584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:45.675133 8584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:45.679168 8584 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44977
I20260812 05:19:45.679256 8653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44977 every 8 connection(s)
I20260812 05:19:45.679620 8584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:45.680473 8654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:45.682812 8654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae: Bootstrap starting.
I20260812 05:19:45.683420 8654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:45.683719 8654 log.cc:824] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:45.684433 8654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae: No bootstrap required, opened a new log
I20260812 05:19:45.686167 8654 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260812 05:19:45.686305 8654 raft_consensus.cc:374] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:45.686328 8654 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97831c738da4c9c8c7d2a1a2a01cdae, State: Initialized, Role: FOLLOWER
I20260812 05:19:45.686522 8654 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260812 05:19:45.686606 8654 raft_consensus.cc:388] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:45.686661 8654 raft_consensus.cc:482] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:45.686712 8654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:45.687255 8654 raft_consensus.cc:504] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260812 05:19:45.687374 8654 leader_election.cc:302] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97831c738da4c9c8c7d2a1a2a01cdae; no voters:
I20260812 05:19:45.687572 8654 leader_election.cc:288] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:45.687635 8657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:45.687813 8657 raft_consensus.cc:686] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [term 1 LEADER]: Becoming Leader. State: Replica: f97831c738da4c9c8c7d2a1a2a01cdae, State: Running, Role: LEADER
I20260812 05:19:45.687865 8654 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:45.687902 8657 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } }
I20260812 05:19:45.688309 8660 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97831c738da4c9c8c7d2a1a2a01cdae. Latest consensus state: current_term: 1 leader_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } } }
I20260812 05:19:45.688405 8660 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:45.688417 8659 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97831c738da4c9c8c7d2a1a2a01cdae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44977 } } }
I20260812 05:19:45.688482 8659 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:45.688622 8667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:45.689152 8667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:45.689674 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8584
I20260812 05:19:45.689818 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:45.690040 19099 external_mini_cluster.cc:1428] Setting key d2027ce960fe7768b5c1361083d9e236
I20260812 05:19:45.691159 8667 catalog_manager.cc:1347] Generated new cluster ID: bcc2be4eb84548d29eac933768d2614d
I20260812 05:19:45.691216 8667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:45.702059 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:45.690486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45023 (local address 127.0.0.1:44977)
0812 05:19:45.697531 (+ 7045us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:45.697539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:45.697572 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:19:45.697679 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:45.697684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:45.697730 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:45.697902 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:45.698134 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:45.699116 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:45.699944 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:45.700384 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:45.700523 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:45.701273 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:45.701277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:45.701277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:45.701539 (+ 262us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:45.701572 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:45.701584 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:45.701659 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:45.701741 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 05:19:45.701808 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6982,"thread_start_us":79,"threads_started":1}
I20260812 05:19:45.703048 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:45.707607 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:45.704488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42651 (local address 127.0.0.1:44977)
0812 05:19:45.704527 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:45.704529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:45.704660 (+ 131us) server_negotiation.cc:407] Connection header received
0812 05:19:45.704703 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:45.704704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:45.704715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:45.704772 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:45.704897 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:45.705448 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:45.706136 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:45.706356 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:45.706446 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:45.707301 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:45.707302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:45.707302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:45.707307 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:45.707322 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:45.707327 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:45.707378 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:45.707485 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 05:19:45.707508 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:45.707624 8667 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:45.708323 8667 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:45.720844 8667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f97831c738da4c9c8c7d2a1a2a01cdae: Generated new TSK 0
I20260812 05:19:45.721050 8667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:45.728781 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8584
2026-08-12T05:19:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:45 dist-test-slave-srgv krb5kdc[8686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:45 dist-test-slave-srgv krb5kdc[8686](info): set up 2 sockets
Aug 12 05:19:45 dist-test-slave-srgv krb5kdc[8686](info): commencing operation
krb5kdc: starting...
W20260812 05:19:46.798882 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:46 dist-test-slave-srgv krb5kdc[8686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:46.924158 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:47.006218 8700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:47.006443 8700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:47.006467 8700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:47.007611 8700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:47.007650 8700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:47.007664 8700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:47.007678 8700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:47.007689 8700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:47.009166 8700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:47.009371 8700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:47.009668 8700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:47.014623 8707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:47.014598 8709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:47.014825 8700 server_base.cc:1034] running on GCE node
W20260812 05:19:47.014698 8706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:47.015133 8700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:47.015437 8700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:47.016628 8700 hybrid_clock.cc:648] HybridClock initialized: now 1786511987016604 us; error 38 us; skew 500 ppm
Aug 12 05:19:47 dist-test-slave-srgv krb5kdc[8686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:47.019255 8700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:47.019767 8700 webserver.cc:460] Webserver started at http://127.0.0.1:37663/ using document root <none> and password file <none>
I20260812 05:19:47.020023 8700 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:47.020094 8700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:47.020249 8700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:47.021337 8700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "60eb3fd483364ce8b01ed72319e36761"
format_stamp: "Formatted at 2026-08-12 05:19:47 on dist-test-slave-srgv"
server_key: "bac2e639b83e9271fe2b627c6f490449"
server_key_iv: "430da3f182f3f2c848355a430483fba6"
server_key_version: "encryptionkey@0"
I20260812 05:19:47.021783 8700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "60eb3fd483364ce8b01ed72319e36761"
format_stamp: "Formatted at 2026-08-12 05:19:47 on dist-test-slave-srgv"
server_key: "bac2e639b83e9271fe2b627c6f490449"
server_key_iv: "430da3f182f3f2c848355a430483fba6"
server_key_version: "encryptionkey@0"
I20260812 05:19:47.023177 8700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:47.023943 8716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:47.024129 8700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:47.024235 8700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "60eb3fd483364ce8b01ed72319e36761"
format_stamp: "Formatted at 2026-08-12 05:19:47 on dist-test-slave-srgv"
server_key: "bac2e639b83e9271fe2b627c6f490449"
server_key_iv: "430da3f182f3f2c848355a430483fba6"
server_key_version: "encryptionkey@0"
I20260812 05:19:47.024318 8700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:47.027788 8700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:47.030292 8700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:47.030467 8700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:47.034795 8700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42877
I20260812 05:19:47.034829 8772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42877 every 8 connection(s)
I20260812 05:19:47.035181 8700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:47.036065 8773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:47.038489 8773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761: Bootstrap starting.
I20260812 05:19:47.039080 8773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:47.039451 8773 log.cc:824] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:47.039561 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8700
I20260812 05:19:47.039701 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:47.039958 19099 external_mini_cluster.cc:1428] Setting key 90e8cc139214b85bd401485645632e63
I20260812 05:19:47.040302 8773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761: No bootstrap required, opened a new log
I20260812 05:19:47.042482 8773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60eb3fd483364ce8b01ed72319e36761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42877 } }
I20260812 05:19:47.042619 8773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:47.042641 8773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60eb3fd483364ce8b01ed72319e36761, State: Initialized, Role: FOLLOWER
I20260812 05:19:47.042814 8773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60eb3fd483364ce8b01ed72319e36761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42877 } }
I20260812 05:19:47.042958 8773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:47.043025 8773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:47.043081 8773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:47.043322 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:47.040466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47359 (local address 127.0.0.1:42877)
0812 05:19:47.040646 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:47.040656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:47.040690 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:19:47.041597 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:47.041606 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:47.041635 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:47.041790 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:47.042535 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:47.042543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:47.042543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:47.042547 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:47.042587 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:47.042597 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:47.042691 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:47.042821 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 05:19:47.042864 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260812 05:19:47.043907 8773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60eb3fd483364ce8b01ed72319e36761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42877 } }
I20260812 05:19:47.044041 8773 leader_election.cc:302] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60eb3fd483364ce8b01ed72319e36761; no voters:
I20260812 05:19:47.044267 8773 leader_election.cc:288] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:47.044294 8778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:47.044363 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:47.044409 8778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [term 1 LEADER]: Becoming Leader. State: Replica: 60eb3fd483364ce8b01ed72319e36761, State: Running, Role: LEADER
I20260812 05:19:47.044494 8778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60eb3fd483364ce8b01ed72319e36761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42877 } }
I20260812 05:19:47.044548 8773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:47.044826 8779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60eb3fd483364ce8b01ed72319e36761. Latest consensus state: current_term: 1 leader_uuid: "60eb3fd483364ce8b01ed72319e36761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60eb3fd483364ce8b01ed72319e36761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42877 } } }
I20260812 05:19:47.044826 8780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60eb3fd483364ce8b01ed72319e36761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60eb3fd483364ce8b01ed72319e36761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42877 } } }
I20260812 05:19:47.044938 8779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:47.044951 8780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:47.045228 8783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:47.045773 8783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:47.047480 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:47.046027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36069 (local address 127.0.0.1:42877)
0812 05:19:47.046067 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:47.046069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:47.046330 (+ 261us) server_negotiation.cc:407] Connection header received
0812 05:19:47.046439 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:47.046440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:47.046445 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:47.046480 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:47.047227 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:47.047228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:47.047229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:47.047233 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:47.047249 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:47.047254 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:47.047283 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:47.047405 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 05:19:47.047420 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:47.047542 8783 catalog_manager.cc:1347] Generated new cluster ID: e9fa4ca59ecc49db9061440d64bf9d18
I20260812 05:19:47.047582 8783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:47.058063 8783 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:47.058269 8783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:47.069734 8783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60eb3fd483364ce8b01ed72319e36761: Generated new TSK 0
I20260812 05:19:47.069943 8783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:47.071972 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8700
2026-08-12T05:19:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:47 dist-test-slave-srgv krb5kdc[8804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:47 dist-test-slave-srgv krb5kdc[8804](info): set up 2 sockets
Aug 12 05:19:47 dist-test-slave-srgv krb5kdc[8804](info): commencing operation
krb5kdc: starting...
W20260812 05:19:48.135079 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:48 dist-test-slave-srgv krb5kdc[8804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:48.260641 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:48.343816 8818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:48.344098 8818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:48.344141 8818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:48.345292 8818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:48.345376 8818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:48.345402 8818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:48.345425 8818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:48.345479 8818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:48.346961 8818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:48.347157 8818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:48.347329 8818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:19:48.352238 8818 server_base.cc:1034] running on GCE node
W20260812 05:19:48.352212 8824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:48.352205 8823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:48.352206 8826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:48.352700 8818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:48.353058 8818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:48.354207 8818 hybrid_clock.cc:648] HybridClock initialized: now 1786511988354194 us; error 29 us; skew 500 ppm
Aug 12 05:19:48 dist-test-slave-srgv krb5kdc[8804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:48.356714 8818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:48.357188 8818 webserver.cc:460] Webserver started at http://127.0.0.1:33795/ using document root <none> and password file <none>
I20260812 05:19:48.357391 8818 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:48.357512 8818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:48.357614 8818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:48.358649 8818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "82c63a6cc06e48a2a10a90bf183d77fa"
format_stamp: "Formatted at 2026-08-12 05:19:48 on dist-test-slave-srgv"
server_key: "814bde5740baa80abdaacb80121c16d2"
server_key_iv: "e788800258ea5d8cfb4988f537e18135"
server_key_version: "encryptionkey@0"
I20260812 05:19:48.358956 8818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "82c63a6cc06e48a2a10a90bf183d77fa"
format_stamp: "Formatted at 2026-08-12 05:19:48 on dist-test-slave-srgv"
server_key: "814bde5740baa80abdaacb80121c16d2"
server_key_iv: "e788800258ea5d8cfb4988f537e18135"
server_key_version: "encryptionkey@0"
I20260812 05:19:48.360268 8818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:48.360968 8833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:48.361163 8818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:48.361276 8818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "82c63a6cc06e48a2a10a90bf183d77fa"
format_stamp: "Formatted at 2026-08-12 05:19:48 on dist-test-slave-srgv"
server_key: "814bde5740baa80abdaacb80121c16d2"
server_key_iv: "e788800258ea5d8cfb4988f537e18135"
server_key_version: "encryptionkey@0"
I20260812 05:19:48.361363 8818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:48.376487 8818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:48.378647 8818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:48.378839 8818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:48.383287 8818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34891
I20260812 05:19:48.383334 8894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34891 every 8 connection(s)
I20260812 05:19:48.383736 8818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:48.384632 8895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:48.386029 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8818
I20260812 05:19:48.386168 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:48.386404 19099 external_mini_cluster.cc:1428] Setting key ab61f47d6a9082209780e1aa38363cf8
I20260812 05:19:48.387691 8895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa: Bootstrap starting.
I20260812 05:19:48.388336 8895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:48.388645 8895 log.cc:824] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:48.389390 8895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa: No bootstrap required, opened a new log
I20260812 05:19:48.391271 8895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34891 } }
I20260812 05:19:48.391477 8895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:48.391522 8895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c63a6cc06e48a2a10a90bf183d77fa, State: Initialized, Role: FOLLOWER
I20260812 05:19:48.391655 8895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34891 } }
I20260812 05:19:48.391741 8895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:48.391765 8895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:48.391855 8895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:48.392457 8895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34891 } }
I20260812 05:19:48.392546 8895 leader_election.cc:302] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c63a6cc06e48a2a10a90bf183d77fa; no voters:
I20260812 05:19:48.392757 8895 leader_election.cc:288] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:48.392781 8900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:48.392912 8900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [term 1 LEADER]: Becoming Leader. State: Replica: 82c63a6cc06e48a2a10a90bf183d77fa, State: Running, Role: LEADER
I20260812 05:19:48.393057 8900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34891 } }
I20260812 05:19:48.393056 8895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 05:19:48 dist-test-slave-srgv krb5kdc[8804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:19:48.393556 8902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34891 } } }
I20260812 05:19:48.393628 8902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:48.393838 8904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:48.394380 8904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:48.394924 8901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c63a6cc06e48a2a10a90bf183d77fa. Latest consensus state: current_term: 1 leader_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c63a6cc06e48a2a10a90bf183d77fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34891 } } }
I20260812 05:19:48.395219 8901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:48.396759 8904 catalog_manager.cc:1347] Generated new cluster ID: c4f7f97863504d29a00dfa5bb1727695
I20260812 05:19:48.396816 8904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:48.398327 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:48.386993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:42053 (local address 127.0.0.1:34891)
0812 05:19:48.387196 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:48.387204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:48.387230 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:19:48.387664 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:48.387671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:48.387721 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:48.387924 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:48.388628 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:48.389728 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:48.390529 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:48.390938 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:48.391065 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:48.394183 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:48.394189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:48.394189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:48.394203 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:48.396366 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:48.396903 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:48.396907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:48.396907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:48.397027 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:48.397304 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:48.397305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:48.397305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:48.397656 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:48.397820 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:48.397968 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 05:19:48.398111 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260812 05:19:48.399377 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:48.408999 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:48.401086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:44315 (local address 127.0.0.1:34891)
0812 05:19:48.401118 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:48.401120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:48.401128 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:19:48.401193 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:48.401193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:48.401203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:48.401256 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:48.402031 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:48.402776 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:48.403865 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:48.404197 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:48.404260 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:48.405850 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:48.405851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:48.405851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:48.405859 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:48.407504 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:48.407952 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:48.407952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:48.407953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:48.408011 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:48.408255 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:48.408256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:48.408256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:48.408393 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:48.408486 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:48.408720 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 05:19:48.408789 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 05:19:48.415454 8904 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:48.416025 8904 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:48.421483 8904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82c63a6cc06e48a2a10a90bf183d77fa: Generated new TSK 0
I20260812 05:19:48.421684 8904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:48.429277 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8818
W20260812 05:19:48.434875 8802 connection.cc:535] client connection to 127.0.0.1:34891 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34891): Connection reset by peer (error 104)
2026-08-12T05:19:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:48 dist-test-slave-srgv krb5kdc[8928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:48 dist-test-slave-srgv krb5kdc[8928](info): set up 2 sockets
Aug 12 05:19:48 dist-test-slave-srgv krb5kdc[8928](info): commencing operation
krb5kdc: starting...
W20260812 05:19:49.496980 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:49 dist-test-slave-srgv krb5kdc[8928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:49.626497 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:49.710388 8942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:49.710690 8942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:49.710737 8942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:49.711959 8942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:49.712044 8942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:49.712070 8942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:49.712098 8942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:49.712127 8942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:49.713686 8942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:49.713994 8942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:49.714255 8942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:49.719363 8947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:49.719363 8948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:49.719573 8942 server_base.cc:1034] running on GCE node
W20260812 05:19:49.719421 8951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:49.719926 8942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:49.720234 8942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:49.721436 8942 hybrid_clock.cc:648] HybridClock initialized: now 1786511989721389 us; error 46 us; skew 500 ppm
Aug 12 05:19:49 dist-test-slave-srgv krb5kdc[8928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:49.724031 8942 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:49.724512 8942 webserver.cc:460] Webserver started at http://127.0.0.1:45169/ using document root <none> and password file <none>
I20260812 05:19:49.724758 8942 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:49.724829 8942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:49.724982 8942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:49.726151 8942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "06260148cf0b4db59d691d193a51f334"
format_stamp: "Formatted at 2026-08-12 05:19:49 on dist-test-slave-srgv"
server_key: "fe27f6ec1dca5881197f9c9fd545754a"
server_key_iv: "e86fe932842ed0484ee44611847ed221"
server_key_version: "encryptionkey@0"
I20260812 05:19:49.726529 8942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "06260148cf0b4db59d691d193a51f334"
format_stamp: "Formatted at 2026-08-12 05:19:49 on dist-test-slave-srgv"
server_key: "fe27f6ec1dca5881197f9c9fd545754a"
server_key_iv: "e86fe932842ed0484ee44611847ed221"
server_key_version: "encryptionkey@0"
I20260812 05:19:49.727933 8942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:19:49.728736 8958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:49.728935 8942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:49.729035 8942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "06260148cf0b4db59d691d193a51f334"
format_stamp: "Formatted at 2026-08-12 05:19:49 on dist-test-slave-srgv"
server_key: "fe27f6ec1dca5881197f9c9fd545754a"
server_key_iv: "e86fe932842ed0484ee44611847ed221"
server_key_version: "encryptionkey@0"
I20260812 05:19:49.729117 8942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:49.739864 8942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:49.742089 8942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:49.742277 8942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:49.746663 8942 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42443
I20260812 05:19:49.746702 9012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42443 every 8 connection(s)
I20260812 05:19:49.747066 8942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:49.748010 9013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:49.750590 9013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334: Bootstrap starting.
I20260812 05:19:49.751217 9013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:49.751526 9013 log.cc:824] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:49.751799 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 8942
I20260812 05:19:49.751946 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:49.752189 19099 external_mini_cluster.cc:1428] Setting key d40ddcc637e072ab3355b6b5ff6f5f60
I20260812 05:19:49.752420 9013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334: No bootstrap required, opened a new log
I20260812 05:19:49.754361 9013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06260148cf0b4db59d691d193a51f334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260812 05:19:49.754499 9013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:49.754522 9013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06260148cf0b4db59d691d193a51f334, State: Initialized, Role: FOLLOWER
I20260812 05:19:49.754669 9013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06260148cf0b4db59d691d193a51f334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260812 05:19:49.754761 9013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:49.754799 9013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:49.754848 9013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:49.755443 9013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06260148cf0b4db59d691d193a51f334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260812 05:19:49.755529 9013 leader_election.cc:302] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06260148cf0b4db59d691d193a51f334; no voters:
I20260812 05:19:49.755728 9013 leader_election.cc:288] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:49.755786 9018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:49.755934 9018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [term 1 LEADER]: Becoming Leader. State: Replica: 06260148cf0b4db59d691d193a51f334, State: Running, Role: LEADER
I20260812 05:19:49.755995 9013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:49.756072 9018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06260148cf0b4db59d691d193a51f334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } }
I20260812 05:19:49.756474 9018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06260148cf0b4db59d691d193a51f334. Latest consensus state: current_term: 1 leader_uuid: "06260148cf0b4db59d691d193a51f334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06260148cf0b4db59d691d193a51f334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260812 05:19:49.756486 9019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06260148cf0b4db59d691d193a51f334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06260148cf0b4db59d691d193a51f334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42443 } } }
I20260812 05:19:49.756630 9018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:49.756642 9019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:49.756881 9028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:49.757439 9028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:49.759857 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:49.752791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:44389 (local address 127.0.0.1:42443)
0812 05:19:49.753015 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:49.753029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:49.753063 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:19:49.754362 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:49.754371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:49.754437 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:49.754642 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:49.754929 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:49.756520 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:49.757727 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:49.758276 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:49.758428 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:49.759214 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:49.759218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:49.759219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:49.759414 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:49.759444 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:49.759455 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:49.759556 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:49.759649 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 05:19:49.759678 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260812 05:19:49.761071 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:49.762620 9028 catalog_manager.cc:1347] Generated new cluster ID: 8404f8527d0b40e7aa9f5eea9c61bc00
I20260812 05:19:49.762676 9028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:49.765894 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:49.762856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:55071 (local address 127.0.0.1:42443)
0812 05:19:49.762895 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:49.762896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:49.762994 (+ 98us) server_negotiation.cc:407] Connection header received
0812 05:19:49.763071 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:49.763071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:49.763082 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:49.763128 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:49.763292 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:49.763809 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:49.764546 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:49.764781 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:49.764858 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:49.765610 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:49.765612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:49.765612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:49.765617 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:49.765631 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:49.765636 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:49.765696 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:49.765782 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:19:49.765797 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:19:49.770622 9028 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:49.771307 9028 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:49.780653 9028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06260148cf0b4db59d691d193a51f334: Generated new TSK 0
I20260812 05:19:49.780872 9028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:49.785663 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 8942
W20260812 05:19:49.791491 8926 connection.cc:535] client connection to 127.0.0.1:42443 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42443): Connection reset by peer (error 104)
2026-08-12T05:19:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:49 dist-test-slave-srgv krb5kdc[9045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:49 dist-test-slave-srgv krb5kdc[9045](info): set up 2 sockets
Aug 12 05:19:49 dist-test-slave-srgv krb5kdc[9045](info): commencing operation
krb5kdc: starting...
W20260812 05:19:50.854534 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:50 dist-test-slave-srgv krb5kdc[9045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:50.978618 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:51.062474 9058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:51.062693 9058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:51.062717 9058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:51.063997 9058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:51.064064 9058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:51.064083 9058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:51.064141 9058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:51.064160 9058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:51.065750 9058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:51.066031 9058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:51.066277 9058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:51.071357 9064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:51.071398 9065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:51.071357 9067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:51.071507 9058 server_base.cc:1034] running on GCE node
I20260812 05:19:51.071810 9058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:51.072075 9058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:51.073254 9058 hybrid_clock.cc:648] HybridClock initialized: now 1786511991073250 us; error 41 us; skew 500 ppm
Aug 12 05:19:51 dist-test-slave-srgv krb5kdc[9045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:51.079272 9058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:51.079831 9058 webserver.cc:460] Webserver started at http://127.0.0.1:38677/ using document root <none> and password file <none>
I20260812 05:19:51.080096 9058 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:51.080166 9058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:51.080327 9058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:51.081501 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "73653f54825a48bba785eca70fcdfc39"
format_stamp: "Formatted at 2026-08-12 05:19:51 on dist-test-slave-srgv"
server_key: "0a61985f1df10a2ef1ddad64a6f45704"
server_key_iv: "1353a61decd72ee9379d96d98fee62bd"
server_key_version: "encryptionkey@0"
I20260812 05:19:51.081893 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "73653f54825a48bba785eca70fcdfc39"
format_stamp: "Formatted at 2026-08-12 05:19:51 on dist-test-slave-srgv"
server_key: "0a61985f1df10a2ef1ddad64a6f45704"
server_key_iv: "1353a61decd72ee9379d96d98fee62bd"
server_key_version: "encryptionkey@0"
I20260812 05:19:51.083331 9058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:19:51.084115 9074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:51.084306 9058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:51.084420 9058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "73653f54825a48bba785eca70fcdfc39"
format_stamp: "Formatted at 2026-08-12 05:19:51 on dist-test-slave-srgv"
server_key: "0a61985f1df10a2ef1ddad64a6f45704"
server_key_iv: "1353a61decd72ee9379d96d98fee62bd"
server_key_version: "encryptionkey@0"
I20260812 05:19:51.084502 9058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:51.088145 9058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:51.090255 9058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:51.090432 9058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:51.094647 9058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46771
I20260812 05:19:51.094688 9128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46771 every 8 connection(s)
I20260812 05:19:51.095041 9058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:51.095949 9129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:51.098376 9129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39: Bootstrap starting.
I20260812 05:19:51.098928 9129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:51.099187 9129 log.cc:824] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:51.099889 9129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39: No bootstrap required, opened a new log
I20260812 05:19:51.101625 9129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73653f54825a48bba785eca70fcdfc39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46771 } }
I20260812 05:19:51.101765 9129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:51.101787 9129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73653f54825a48bba785eca70fcdfc39, State: Initialized, Role: FOLLOWER
I20260812 05:19:51.101889 9129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73653f54825a48bba785eca70fcdfc39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46771 } }
I20260812 05:19:51.101943 9129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:51.101962 9129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:51.102036 9129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:51.102599 9129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73653f54825a48bba785eca70fcdfc39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46771 } }
I20260812 05:19:51.102720 9129 leader_election.cc:302] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73653f54825a48bba785eca70fcdfc39; no voters:
I20260812 05:19:51.102907 9129 leader_election.cc:288] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:51.102950 9132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:51.103142 9132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [term 1 LEADER]: Becoming Leader. State: Replica: 73653f54825a48bba785eca70fcdfc39, State: Running, Role: LEADER
I20260812 05:19:51.103188 9129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:51.103272 9132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73653f54825a48bba785eca70fcdfc39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46771 } }
I20260812 05:19:51.103786 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73653f54825a48bba785eca70fcdfc39. Latest consensus state: current_term: 1 leader_uuid: "73653f54825a48bba785eca70fcdfc39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73653f54825a48bba785eca70fcdfc39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46771 } } }
I20260812 05:19:51.103873 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:51.103956 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9058
I20260812 05:19:51.104086 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:51.104341 9136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:51.104377 19099 external_mini_cluster.cc:1428] Setting key 204bb27537db2004dbf7874e8cde7d2e
I20260812 05:19:51.104622 9133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73653f54825a48bba785eca70fcdfc39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73653f54825a48bba785eca70fcdfc39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46771 } } }
I20260812 05:19:51.104768 9133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:51.104892 9136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:51.106689 9136 catalog_manager.cc:1347] Generated new cluster ID: 58f6552d519b41bfa3a04dd164758aa9
I20260812 05:19:51.106747 9136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:51.107064 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:51.105065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:55657 (local address 127.0.0.1:46771)
0812 05:19:51.105274 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:51.105284 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:51.105482 (+ 198us) server_negotiation.cc:407] Connection header received
0812 05:19:51.105666 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:51.105676 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:51.105710 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:51.105853 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:51.106575 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:51.106578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:51.106578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:51.106582 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:51.106660 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:51.106676 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:51.106742 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:51.106853 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 05:19:51.106899 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260812 05:19:51.108083 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:51.110845 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:51.109711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:49279 (local address 127.0.0.1:46771)
0812 05:19:51.109839 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:51.109841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:51.109850 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:19:51.109903 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:51.109904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:51.109910 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:51.109941 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:51.110602 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:51.110603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:51.110603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:51.110607 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:51.110623 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:51.110627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:51.110694 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:51.110772 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 05:19:51.110786 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260812 05:19:51.119902 9136 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:51.120116 9136 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:51.125484 9136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73653f54825a48bba785eca70fcdfc39: Generated new TSK 0
I20260812 05:19:51.125723 9136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:51.132222 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9058
2026-08-12T05:19:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:51 dist-test-slave-srgv krb5kdc[9161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:51 dist-test-slave-srgv krb5kdc[9161](info): set up 2 sockets
Aug 12 05:19:51 dist-test-slave-srgv krb5kdc[9161](info): commencing operation
krb5kdc: starting...
W20260812 05:19:52.222487 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:52 dist-test-slave-srgv krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:52.344647 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:52.428187 9176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:52.428474 9176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:52.428519 9176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:52.429787 9176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:52.429857 9176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:52.429881 9176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:52.429909 9176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:52.429936 9176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:52.431456 9176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:52.431756 9176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:52.431994 9176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:52.437184 9181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:52.437167 9182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:52.437135 9185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:52.437223 9176 server_base.cc:1034] running on GCE node
I20260812 05:19:52.437727 9176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:52.438032 9176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:52.439209 9176 hybrid_clock.cc:648] HybridClock initialized: now 1786511992439198 us; error 42 us; skew 500 ppm
Aug 12 05:19:52 dist-test-slave-srgv krb5kdc[9161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:52.441803 9176 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:52.442266 9176 webserver.cc:460] Webserver started at http://127.0.0.1:36393/ using document root <none> and password file <none>
I20260812 05:19:52.442502 9176 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:52.442572 9176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:52.442734 9176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:52.443866 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "706c8684469a4b7eacc13eb2fa3e073c"
format_stamp: "Formatted at 2026-08-12 05:19:52 on dist-test-slave-srgv"
server_key: "393b753bfa8c6b4d5be497be6e2be52f"
server_key_iv: "b5420eb3e9539fdebfeaad0ddaf8adab"
server_key_version: "encryptionkey@0"
I20260812 05:19:52.444280 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "706c8684469a4b7eacc13eb2fa3e073c"
format_stamp: "Formatted at 2026-08-12 05:19:52 on dist-test-slave-srgv"
server_key: "393b753bfa8c6b4d5be497be6e2be52f"
server_key_iv: "b5420eb3e9539fdebfeaad0ddaf8adab"
server_key_version: "encryptionkey@0"
I20260812 05:19:52.445737 9176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:19:52.446472 9193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:52.446659 9176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:52.446770 9176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "706c8684469a4b7eacc13eb2fa3e073c"
format_stamp: "Formatted at 2026-08-12 05:19:52 on dist-test-slave-srgv"
server_key: "393b753bfa8c6b4d5be497be6e2be52f"
server_key_iv: "b5420eb3e9539fdebfeaad0ddaf8adab"
server_key_version: "encryptionkey@0"
I20260812 05:19:52.446846 9176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:52.462636 9176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:52.465142 9176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:52.465329 9176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:52.469305 9176 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46105
I20260812 05:19:52.469362 9248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46105 every 8 connection(s)
I20260812 05:19:52.469763 9176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:52.470151 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9176
I20260812 05:19:52.470276 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:52.470461 19099 external_mini_cluster.cc:1428] Setting key 13115f11d0a6416771cebd944401cf05
I20260812 05:19:52.470841 9249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:52.474485 9249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c: Bootstrap starting.
I20260812 05:19:52.475212 9249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:52.475519 9249 log.cc:824] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:52.476334 9249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c: No bootstrap required, opened a new log
Aug 12 05:19:52 dist-test-slave-srgv krb5kdc[9161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:19:52.478721 9249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706c8684469a4b7eacc13eb2fa3e073c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260812 05:19:52.478914 9249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:52.478957 9249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706c8684469a4b7eacc13eb2fa3e073c, State: Initialized, Role: FOLLOWER
I20260812 05:19:52.479108 9249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706c8684469a4b7eacc13eb2fa3e073c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260812 05:19:52.479205 9249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:52.479245 9249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:52.479301 9249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:52.480103 9249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706c8684469a4b7eacc13eb2fa3e073c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260812 05:19:52.480222 9249 leader_election.cc:302] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 706c8684469a4b7eacc13eb2fa3e073c; no voters:
I20260812 05:19:52.480400 9249 leader_election.cc:288] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:52.480476 9254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:52.480683 9254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [term 1 LEADER]: Becoming Leader. State: Replica: 706c8684469a4b7eacc13eb2fa3e073c, State: Running, Role: LEADER
I20260812 05:19:52.480698 9249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:52.480856 9254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706c8684469a4b7eacc13eb2fa3e073c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } }
I20260812 05:19:52.481252 9255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "706c8684469a4b7eacc13eb2fa3e073c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706c8684469a4b7eacc13eb2fa3e073c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } } }
I20260812 05:19:52.481336 9255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:52.481578 9256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706c8684469a4b7eacc13eb2fa3e073c. Latest consensus state: current_term: 1 leader_uuid: "706c8684469a4b7eacc13eb2fa3e073c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706c8684469a4b7eacc13eb2fa3e073c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46105 } } }
I20260812 05:19:52.481822 9256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:52.481995 9260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:52.482643 9260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:52.483000 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:52.471017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:49657 (local address 127.0.0.1:46105)
0812 05:19:52.471228 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:52.471236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:52.471262 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:19:52.471695 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:52.471703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:52.471780 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:52.471911 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:52.472843 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:52.473891 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:52.474968 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:52.475488 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:52.475583 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:52.478299 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:52.478306 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:52.478306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:52.478323 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:52.480487 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:52.481032 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:52.481037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:52.481037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:52.481157 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:52.481454 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:52.481456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:52.481456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:52.481805 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:52.481984 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:52.482526 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 05:19:52.482634 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260812 05:19:52.483978 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:52.485498 9260 catalog_manager.cc:1347] Generated new cluster ID: d36e1a8e8777422ab68141a63be819d0
I20260812 05:19:52.485596 9260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:52.496042 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:52.485674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:46259 (local address 127.0.0.1:46105)
0812 05:19:52.485712 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:52.485714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:52.489714 (+ 4000us) server_negotiation.cc:407] Connection header received
0812 05:19:52.489792 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:52.489793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:52.489805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:52.489857 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:52.490431 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:52.490957 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:52.491685 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:52.491912 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:52.491989 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:52.493266 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:52.493268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:52.493268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:52.493277 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:52.494815 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:52.495198 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:52.495200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:52.495205 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:52.495272 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:52.495506 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:52.495507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:52.495507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:52.495628 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:52.495747 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:52.495865 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:19:52.495931 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:52.501214 9260 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:52.501984 9260 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:52.524515 9260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 706c8684469a4b7eacc13eb2fa3e073c: Generated new TSK 0
I20260812 05:19:52.524910 9260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:52.538034 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9176
W20260812 05:19:52.543902 9158 connection.cc:535] client connection to 127.0.0.1:46105 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46105): Connection reset by peer (error 104)
2026-08-12T05:19:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:52 dist-test-slave-srgv krb5kdc[9282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:52 dist-test-slave-srgv krb5kdc[9282](info): set up 2 sockets
Aug 12 05:19:52 dist-test-slave-srgv krb5kdc[9282](info): commencing operation
krb5kdc: starting...
W20260812 05:19:53.607833 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:53 dist-test-slave-srgv krb5kdc[9282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:53.732712 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46707
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:53.817016 9295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:53.817299 9295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:53.817340 9295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:53.818591 9295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:53.818675 9295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:53.818699 9295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:53.818720 9295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:53.818750 9295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:53.820276 9295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46707
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:53.820580 9295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:53.820859 9295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:53.825914 9300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:53.825912 9303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:53.825913 9301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:53.825971 9295 server_base.cc:1034] running on GCE node
I20260812 05:19:53.826438 9295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:53.826771 9295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:53.827955 9295 hybrid_clock.cc:648] HybridClock initialized: now 1786511993827950 us; error 38 us; skew 500 ppm
Aug 12 05:19:53 dist-test-slave-srgv krb5kdc[9282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:53.830646 9295 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:53.831103 9295 webserver.cc:460] Webserver started at http://127.0.0.1:42851/ using document root <none> and password file <none>
I20260812 05:19:53.831362 9295 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:53.831435 9295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:53.831593 9295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:53.832683 9295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "5febc4861c7d4ebeaac84c5e6230764f"
format_stamp: "Formatted at 2026-08-12 05:19:53 on dist-test-slave-srgv"
server_key: "3fbb0a114582c89b246c09df07b6c978"
server_key_iv: "807855affbadf3af62ddfa4e8247fec2"
server_key_version: "encryptionkey@0"
I20260812 05:19:53.833056 9295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "5febc4861c7d4ebeaac84c5e6230764f"
format_stamp: "Formatted at 2026-08-12 05:19:53 on dist-test-slave-srgv"
server_key: "3fbb0a114582c89b246c09df07b6c978"
server_key_iv: "807855affbadf3af62ddfa4e8247fec2"
server_key_version: "encryptionkey@0"
I20260812 05:19:53.834534 9295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:19:53.835309 9310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:53.835493 9295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:19:53.835605 9295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "5febc4861c7d4ebeaac84c5e6230764f"
format_stamp: "Formatted at 2026-08-12 05:19:53 on dist-test-slave-srgv"
server_key: "3fbb0a114582c89b246c09df07b6c978"
server_key_iv: "807855affbadf3af62ddfa4e8247fec2"
server_key_version: "encryptionkey@0"
I20260812 05:19:53.835682 9295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:53.858302 9295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:53.860734 9295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:53.860913 9295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:53.865238 9295 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46707
I20260812 05:19:53.865273 9365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46707 every 8 connection(s)
I20260812 05:19:53.865697 9295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:53.866571 9366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:53.868265 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9295
I20260812 05:19:53.868402 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:53.868615 19099 external_mini_cluster.cc:1428] Setting key 1591203b6fa8e2b10e4623f52d9ce352
I20260812 05:19:53.869095 9366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f: Bootstrap starting.
I20260812 05:19:53.869884 9366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:53.870224 9366 log.cc:824] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:53.871042 9366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f: No bootstrap required, opened a new log
I20260812 05:19:53.873324 9366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5febc4861c7d4ebeaac84c5e6230764f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260812 05:19:53.873536 9366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:53.873579 9366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5febc4861c7d4ebeaac84c5e6230764f, State: Initialized, Role: FOLLOWER
I20260812 05:19:53.873755 9366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5febc4861c7d4ebeaac84c5e6230764f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260812 05:19:53.873843 9366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:53.873867 9366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:53.873957 9366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:53.874615 9366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5febc4861c7d4ebeaac84c5e6230764f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260812 05:19:53.874729 9366 leader_election.cc:302] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5febc4861c7d4ebeaac84c5e6230764f; no voters:
I20260812 05:19:53.874933 9366 leader_election.cc:288] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:53.875109 9372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:53.875169 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:53.869199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:36067 (local address 127.0.0.1:46707)
0812 05:19:53.869433 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:53.869443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:53.869483 (+ 40us) server_negotiation.cc:407] Connection header received
0812 05:19:53.870126 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:53.870136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:53.870207 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:53.870360 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:53.870622 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:53.872014 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:53.872830 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:53.873495 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:53.873613 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:53.874439 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:53.874443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:53.874444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:53.874662 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:53.874703 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:53.874719 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:53.874831 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:53.874922 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 05:19:53.874949 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260812 05:19:53.875187 9366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:53.875290 9372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [term 1 LEADER]: Becoming Leader. State: Replica: 5febc4861c7d4ebeaac84c5e6230764f, State: Running, Role: LEADER
I20260812 05:19:53.875392 9372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5febc4861c7d4ebeaac84c5e6230764f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } }
I20260812 05:19:53.875779 9371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5febc4861c7d4ebeaac84c5e6230764f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5febc4861c7d4ebeaac84c5e6230764f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } } }
I20260812 05:19:53.875873 9371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:53.875847 9373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5febc4861c7d4ebeaac84c5e6230764f. Latest consensus state: current_term: 1 leader_uuid: "5febc4861c7d4ebeaac84c5e6230764f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5febc4861c7d4ebeaac84c5e6230764f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46707 } } }
I20260812 05:19:53.875926 9373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:53.876230 9377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:53.876786 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:53.876952 9377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:53.878697 9377 catalog_manager.cc:1347] Generated new cluster ID: 02a4c660f1004709ae8bb7afa59c549c
I20260812 05:19:53.878754 9377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:53.882412 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:53.878406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:45147 (local address 127.0.0.1:46707)
0812 05:19:53.878441 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:53.878444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:53.878602 (+ 158us) server_negotiation.cc:407] Connection header received
0812 05:19:53.878709 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:53.878710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:53.878722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:53.878784 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:53.878998 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:53.879740 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:53.880962 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:53.881271 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:53.881314 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:53.882139 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:53.882140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:53.882141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:53.882145 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:53.882160 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:53.882165 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:53.882199 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:53.882330 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 05:19:53.882341 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 05:19:53.902532 9377 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:53.903272 9377 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:53.912209 9377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5febc4861c7d4ebeaac84c5e6230764f: Generated new TSK 0
I20260812 05:19:53.912416 9377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:53.919370 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9295
W20260812 05:19:53.925200 9280 connection.cc:535] client connection to 127.0.0.1:46707 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46707): Connection reset by peer (error 104)
2026-08-12T05:19:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:53 dist-test-slave-srgv krb5kdc[9397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:53 dist-test-slave-srgv krb5kdc[9397](info): set up 2 sockets
Aug 12 05:19:53 dist-test-slave-srgv krb5kdc[9397](info): commencing operation
krb5kdc: starting...
W20260812 05:19:54.998550 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:55 dist-test-slave-srgv krb5kdc[9397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:19:55.125309 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:55.207626 9413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:55.207854 9413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:55.207880 9413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:55.208997 9413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:55.209039 9413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:55.209053 9413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:55.209066 9413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:55.209079 9413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:55.210561 9413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:55.210767 9413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:55.210940 9413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:55.216050 9421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:55.216310 9413 server_base.cc:1034] running on GCE node
W20260812 05:19:55.216071 9418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:55.216063 9419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:55.216624 9413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:55.216989 9413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:55.218174 9413 hybrid_clock.cc:648] HybridClock initialized: now 1786511995218150 us; error 38 us; skew 500 ppm
Aug 12 05:19:55 dist-test-slave-srgv krb5kdc[9397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:55.220834 9413 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:19:55.221315 9413 webserver.cc:460] Webserver started at http://127.0.0.1:46439/ using document root <none> and password file <none>
I20260812 05:19:55.221637 9413 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:55.221714 9413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:55.221871 9413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:55.222994 9413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "1681bac238594cb2a4a52e660aae3ad3"
format_stamp: "Formatted at 2026-08-12 05:19:55 on dist-test-slave-srgv"
server_key: "73ab134e3e8200abe3646d5bf6c0fdfc"
server_key_iv: "c0bace88e5374f62a0e601bb545b03c8"
server_key_version: "encryptionkey@0"
I20260812 05:19:55.223389 9413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "1681bac238594cb2a4a52e660aae3ad3"
format_stamp: "Formatted at 2026-08-12 05:19:55 on dist-test-slave-srgv"
server_key: "73ab134e3e8200abe3646d5bf6c0fdfc"
server_key_iv: "c0bace88e5374f62a0e601bb545b03c8"
server_key_version: "encryptionkey@0"
I20260812 05:19:55.224910 9413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:55.225801 9428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:55.226073 9413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:19:55.226190 9413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "1681bac238594cb2a4a52e660aae3ad3"
format_stamp: "Formatted at 2026-08-12 05:19:55 on dist-test-slave-srgv"
server_key: "73ab134e3e8200abe3646d5bf6c0fdfc"
server_key_iv: "c0bace88e5374f62a0e601bb545b03c8"
server_key_version: "encryptionkey@0"
I20260812 05:19:55.226279 9413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:55.229712 9413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:55.231791 9413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:55.231952 9413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:55.236007 9413 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33753
I20260812 05:19:55.236042 9483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33753 every 8 connection(s)
I20260812 05:19:55.236367 9413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:55.237293 9484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:55.239626 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3: Bootstrap starting.
I20260812 05:19:55.240243 9484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:55.240537 9484 log.cc:824] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:55.240572 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9413
I20260812 05:19:55.240710 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:55.240911 19099 external_mini_cluster.cc:1428] Setting key 5981396414a82a81c94e4771dcead7d6
I20260812 05:19:55.241356 9484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3: No bootstrap required, opened a new log
I20260812 05:19:55.243124 9484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1681bac238594cb2a4a52e660aae3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33753 } }
I20260812 05:19:55.243299 9484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:55.243345 9484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1681bac238594cb2a4a52e660aae3ad3, State: Initialized, Role: FOLLOWER
I20260812 05:19:55.243866 9484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1681bac238594cb2a4a52e660aae3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33753 } }
I20260812 05:19:55.244000 9484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:55.244076 9484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:55.244136 9484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:55.244513 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:55.241482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:46405 (local address 127.0.0.1:33753)
0812 05:19:55.241675 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:55.241690 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:55.241715 (+ 25us) server_negotiation.cc:407] Connection header received
0812 05:19:55.243110 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:55.243116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:55.243145 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:55.243269 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:55.244008 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:55.244012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:55.244013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:55.244017 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:55.244076 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:55.244097 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:55.244182 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:55.244309 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 05:19:55.244358 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260812 05:19:55.245389 9484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1681bac238594cb2a4a52e660aae3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33753 } }
I20260812 05:19:55.245391 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:55.245532 9484 leader_election.cc:302] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1681bac238594cb2a4a52e660aae3ad3; no voters:
I20260812 05:19:55.245692 9484 leader_election.cc:288] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:55.245721 9489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:55.245829 9489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [term 1 LEADER]: Becoming Leader. State: Replica: 1681bac238594cb2a4a52e660aae3ad3, State: Running, Role: LEADER
I20260812 05:19:55.245951 9484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:55.245962 9489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1681bac238594cb2a4a52e660aae3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33753 } }
I20260812 05:19:55.246423 9490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1681bac238594cb2a4a52e660aae3ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1681bac238594cb2a4a52e660aae3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33753 } } }
I20260812 05:19:55.246533 9490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:55.246438 9491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1681bac238594cb2a4a52e660aae3ad3. Latest consensus state: current_term: 1 leader_uuid: "1681bac238594cb2a4a52e660aae3ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1681bac238594cb2a4a52e660aae3ad3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33753 } } }
I20260812 05:19:55.246762 9502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:55.246765 9491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:55.247671 9502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:55.248322 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:55.246994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:44855 (local address 127.0.0.1:33753)
0812 05:19:55.247033 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:55.247035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:55.247167 (+ 132us) server_negotiation.cc:407] Connection header received
0812 05:19:55.247269 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:55.247269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:55.247276 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:55.247306 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:55.247964 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:55.247965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:55.247965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:55.247970 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:55.247983 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:55.247988 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:55.248055 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:55.248223 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 05:19:55.248240 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:55.249271 9502 catalog_manager.cc:1347] Generated new cluster ID: 1232fec5e13b40a7acc7117c82c985cf
I20260812 05:19:55.249326 9502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:55.267808 9502 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:55.268014 9502 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:55.273604 9502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1681bac238594cb2a4a52e660aae3ad3: Generated new TSK 0
I20260812 05:19:55.273829 9502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:55.277091 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9413
2026-08-12T05:19:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:55 dist-test-slave-srgv krb5kdc[9516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:55 dist-test-slave-srgv krb5kdc[9516](info): set up 2 sockets
Aug 12 05:19:55 dist-test-slave-srgv krb5kdc[9516](info): commencing operation
krb5kdc: starting...
W20260812 05:19:56.351687 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:56 dist-test-slave-srgv krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:19:56.481313 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34247
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:34247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:56.566150 9529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:56.566432 9529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:56.566479 9529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:56.567672 9529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:56.567757 9529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:56.567780 9529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:56.567802 9529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:56.567834 9529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:56.569662 9529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:34247
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:56.570003 9529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:56.570281 9529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:56.575311 9537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:56.575322 9535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:56.575361 9534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:56.575595 9529 server_base.cc:1034] running on GCE node
I20260812 05:19:56.575843 9529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:56.576153 9529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:56.577330 9529 hybrid_clock.cc:648] HybridClock initialized: now 1786511996577309 us; error 37 us; skew 500 ppm
Aug 12 05:19:56 dist-test-slave-srgv krb5kdc[9516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511996, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:56.585250 9529 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:56.585801 9529 webserver.cc:460] Webserver started at http://127.18.166.254:35919/ using document root <none> and password file <none>
I20260812 05:19:56.586017 9529 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:56.586063 9529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:56.586179 9529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:56.587402 9529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "afac57ba29d04ca09955b6d2099e9bff"
format_stamp: "Formatted at 2026-08-12 05:19:56 on dist-test-slave-srgv"
server_key: "62fdf3067adb67d6e07878e246e1ded8"
server_key_iv: "038d5d9e9b2024ad6729e1ae9ff76a56"
server_key_version: "encryptionkey@0"
I20260812 05:19:56.587795 9529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "afac57ba29d04ca09955b6d2099e9bff"
format_stamp: "Formatted at 2026-08-12 05:19:56 on dist-test-slave-srgv"
server_key: "62fdf3067adb67d6e07878e246e1ded8"
server_key_iv: "038d5d9e9b2024ad6729e1ae9ff76a56"
server_key_version: "encryptionkey@0"
I20260812 05:19:56.589324 9529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:19:56.590256 9545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:56.590471 9529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:56.590570 9529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "afac57ba29d04ca09955b6d2099e9bff"
format_stamp: "Formatted at 2026-08-12 05:19:56 on dist-test-slave-srgv"
server_key: "62fdf3067adb67d6e07878e246e1ded8"
server_key_iv: "038d5d9e9b2024ad6729e1ae9ff76a56"
server_key_version: "encryptionkey@0"
I20260812 05:19:56.590649 9529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:56.598501 9529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:56.600943 9529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:56.601107 9529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:56.605362 9529 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:34247
I20260812 05:19:56.605463 9599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:34247 every 8 connection(s)
I20260812 05:19:56.605824 9529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:56.606626 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9529
I20260812 05:19:56.606747 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:56.606808 9600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:56.606966 19099 external_mini_cluster.cc:1428] Setting key 48d7d92c50f14dfcca5252c86ccbf4f2
I20260812 05:19:56.610352 9600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff: Bootstrap starting.
I20260812 05:19:56.611047 9600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:56.611356 9600 log.cc:824] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:56.612049 9600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff: No bootstrap required, opened a new log
I20260812 05:19:56.613991 9600 raft_consensus.cc:348] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afac57ba29d04ca09955b6d2099e9bff" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34247 } }
Aug 12 05:19:56 dist-test-slave-srgv krb5kdc[9516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:19:56.614159 9600 raft_consensus.cc:374] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:56.614212 9600 raft_consensus.cc:729] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afac57ba29d04ca09955b6d2099e9bff, State: Initialized, Role: FOLLOWER
I20260812 05:19:56.614346 9600 consensus_queue.cc:260] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afac57ba29d04ca09955b6d2099e9bff" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34247 } }
I20260812 05:19:56.614425 9600 raft_consensus.cc:388] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:56.614470 9600 raft_consensus.cc:482] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:56.614521 9600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:56.615335 9600 raft_consensus.cc:504] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afac57ba29d04ca09955b6d2099e9bff" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34247 } }
I20260812 05:19:56.615459 9600 leader_election.cc:302] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afac57ba29d04ca09955b6d2099e9bff; no voters:
I20260812 05:19:56.615644 9600 leader_election.cc:288] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:56.615705 9605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:56.615901 9605 raft_consensus.cc:686] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [term 1 LEADER]: Becoming Leader. State: Replica: afac57ba29d04ca09955b6d2099e9bff, State: Running, Role: LEADER
I20260812 05:19:56.615976 9600 sys_catalog.cc:564] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:56.616037 9605 consensus_queue.cc:237] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afac57ba29d04ca09955b6d2099e9bff" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34247 } }
I20260812 05:19:56.616415 9608 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader afac57ba29d04ca09955b6d2099e9bff. Latest consensus state: current_term: 1 leader_uuid: "afac57ba29d04ca09955b6d2099e9bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afac57ba29d04ca09955b6d2099e9bff" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34247 } } }
I20260812 05:19:56.616619 9608 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:56.616395 9606 sys_catalog.cc:455] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afac57ba29d04ca09955b6d2099e9bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afac57ba29d04ca09955b6d2099e9bff" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34247 } } }
I20260812 05:19:56.617174 9606 sys_catalog.cc:458] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:56.617489 9615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:56.617942 9615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:56.619885 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:56.607506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34653 (local address 127.18.166.254:34247)
0812 05:19:56.607706 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:56.607716 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:56.607752 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:19:56.608501 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:56.608510 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:56.608576 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:56.608746 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:56.609363 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:56.610779 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:56.611632 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:56.612064 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:56.612184 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:56.614758 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:56.614762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:56.614762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:56.614775 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:56.618027 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:56.618510 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:56.618513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:56.618514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:56.618626 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:56.618901 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:56.618902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:56.618902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:56.619219 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:56.619379 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:56.619581 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 05:19:56.619690 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260812 05:19:56.620476 9615 catalog_manager.cc:1347] Generated new cluster ID: 24bb07f85eec4430b1a0cb9012568691
I20260812 05:19:56.620536 9615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:56.620929 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:56.631987 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:56.622515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41381 (local address 127.18.166.254:34247)
0812 05:19:56.622551 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:56.622553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:56.622669 (+ 116us) server_negotiation.cc:407] Connection header received
0812 05:19:56.622767 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:56.622768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:56.622777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:56.622806 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:56.623592 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:56.624731 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:56.625822 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:56.626155 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:56.626217 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:56.627817 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:56.627818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:56.627818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:19:56.627826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:56.630574 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:56.631035 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:56.631037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:56.631037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:56.631095 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:19:56.631341 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:19:56.631342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:19:56.631342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:19:56.631481 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:56.631564 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:56.631820 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 05:19:56.631893 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:19:56.640185 9615 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:56.640751 9615 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:56.650220 9615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afac57ba29d04ca09955b6d2099e9bff: Generated new TSK 0
I20260812 05:19:56.650465 9615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:56.652427 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9529
W20260812 05:19:56.658742 9514 connection.cc:535] client connection to 127.18.166.254:34247 recv error: Network error: failed to read from TLS socket (remote: 127.18.166.254:34247): Connection reset by peer (error 104)
2026-08-12T05:19:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:56 dist-test-slave-srgv krb5kdc[9632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:56 dist-test-slave-srgv krb5kdc[9632](info): set up 2 sockets
Aug 12 05:19:56 dist-test-slave-srgv krb5kdc[9632](info): commencing operation
krb5kdc: starting...
W20260812 05:19:57.727155 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:57 dist-test-slave-srgv krb5kdc[9632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:19:57.850747 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:44023
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:44023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:57.933311 9645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:57.933570 9645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:57.933606 9645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:57.934731 9645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:57.934805 9645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:57.934832 9645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:57.934864 9645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:57.934888 9645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:57.936535 9645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:44023
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:44023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:57.936806 9645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:57.937033 9645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:19:57.941972 9651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:57.941946 9653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:57.941962 9650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:57.942099 9645 server_base.cc:1034] running on GCE node
I20260812 05:19:57.942479 9645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:57.942803 9645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:57.943985 9645 hybrid_clock.cc:648] HybridClock initialized: now 1786511997943975 us; error 37 us; skew 500 ppm
Aug 12 05:19:57 dist-test-slave-srgv krb5kdc[9632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511997, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:57.947140 9645 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:57.947728 9645 webserver.cc:460] Webserver started at http://127.18.166.254:34735/ using document root <none> and password file <none>
I20260812 05:19:57.948020 9645 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:57.948083 9645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:57.948237 9645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:57.949728 9645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "fd7c349ada0c47408bf451955819e4ea"
format_stamp: "Formatted at 2026-08-12 05:19:57 on dist-test-slave-srgv"
server_key: "ea91c7609db8680d9900242fba961547"
server_key_iv: "040c61b13839b1d8b395e3c9648294bb"
server_key_version: "encryptionkey@0"
I20260812 05:19:57.950146 9645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "fd7c349ada0c47408bf451955819e4ea"
format_stamp: "Formatted at 2026-08-12 05:19:57 on dist-test-slave-srgv"
server_key: "ea91c7609db8680d9900242fba961547"
server_key_iv: "040c61b13839b1d8b395e3c9648294bb"
server_key_version: "encryptionkey@0"
I20260812 05:19:57.952034 9645 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 05:19:57.953044 9660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:57.953222 9645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:19:57.953372 9645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "fd7c349ada0c47408bf451955819e4ea"
format_stamp: "Formatted at 2026-08-12 05:19:57 on dist-test-slave-srgv"
server_key: "ea91c7609db8680d9900242fba961547"
server_key_iv: "040c61b13839b1d8b395e3c9648294bb"
server_key_version: "encryptionkey@0"
I20260812 05:19:57.953472 9645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:57.967929 9645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:57.970039 9645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:57.970218 9645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:57.974570 9645 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:44023
I20260812 05:19:57.974623 9718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:44023 every 8 connection(s)
I20260812 05:19:57.975016 9645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:57.976013 9719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:57.976061 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9645
I20260812 05:19:57.976181 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:57.976421 19099 external_mini_cluster.cc:1428] Setting key c0bbed4ab7924227b32a0e0590bc3f6d
I20260812 05:19:57.979488 9719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea: Bootstrap starting.
I20260812 05:19:57.980247 9719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:57.980556 9719 log.cc:824] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:57.981323 9719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea: No bootstrap required, opened a new log
I20260812 05:19:57.982306 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:57.976969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40953 (local address 127.18.166.254:44023)
0812 05:19:57.977162 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:57.977171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:57.977198 (+ 27us) server_negotiation.cc:407] Connection header received
0812 05:19:57.977762 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:57.977770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:57.977820 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:57.977937 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:57.978200 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:57.979090 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:57.980180 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:57.980627 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:57.980752 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:57.981584 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:57.981588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:57.981588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:57.981802 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:57.981846 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:57.981863 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:57.981954 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:57.982052 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 05:19:57.982080 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260812 05:19:57.983407 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:57.983697 9719 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7c349ada0c47408bf451955819e4ea" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 44023 } }
I20260812 05:19:57.983865 9719 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:57.983903 9719 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd7c349ada0c47408bf451955819e4ea, State: Initialized, Role: FOLLOWER
I20260812 05:19:57.984028 9719 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7c349ada0c47408bf451955819e4ea" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 44023 } }
I20260812 05:19:57.984118 9719 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:57.984149 9719 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:57.984192 9719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:57.985011 9719 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7c349ada0c47408bf451955819e4ea" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 44023 } }
I20260812 05:19:57.985150 9719 leader_election.cc:302] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd7c349ada0c47408bf451955819e4ea; no voters:
I20260812 05:19:57.985374 9719 leader_election.cc:288] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:57.985399 9729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:57.985579 9729 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [term 1 LEADER]: Becoming Leader. State: Replica: fd7c349ada0c47408bf451955819e4ea, State: Running, Role: LEADER
I20260812 05:19:57.985663 9719 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:57.985692 9729 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7c349ada0c47408bf451955819e4ea" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 44023 } }
I20260812 05:19:57.986055 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd7c349ada0c47408bf451955819e4ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7c349ada0c47408bf451955819e4ea" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 44023 } } }
I20260812 05:19:57.986132 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:57.986274 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd7c349ada0c47408bf451955819e4ea. Latest consensus state: current_term: 1 leader_uuid: "fd7c349ada0c47408bf451955819e4ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7c349ada0c47408bf451955819e4ea" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 44023 } } }
I20260812 05:19:57.986326 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:57.986438 9737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:57.986948 9737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:57.989118 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:57.985062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56457 (local address 127.18.166.254:44023)
0812 05:19:57.985145 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:57.985146 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:57.985208 (+ 62us) server_negotiation.cc:407] Connection header received
0812 05:19:57.985294 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:57.985295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:57.985304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:57.985366 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:57.985566 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:57.986652 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:57.987746 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:19:57.988033 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:19:57.988103 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:19:57.988845 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:57.988846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:57.988847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:57.988852 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:57.988868 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:57.988872 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:57.988939 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:57.989012 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 05:19:57.989025 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:19:57.993984 9737 catalog_manager.cc:1347] Generated new cluster ID: 30d4917a73f24068b65bc58338cd0044
I20260812 05:19:57.994057 9737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:58.009198 9737 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:58.009819 9737 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:58.017563 9737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd7c349ada0c47408bf451955819e4ea: Generated new TSK 0
I20260812 05:19:58.017781 9737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:58.026746 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9645
W20260812 05:19:58.032945 9630 connection.cc:535] client connection to 127.18.166.254:44023 recv error: Network error: failed to read from TLS socket (remote: 127.18.166.254:44023): Connection reset by peer (error 104)
2026-08-12T05:19:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:58 dist-test-slave-srgv krb5kdc[9751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:58 dist-test-slave-srgv krb5kdc[9751](info): set up 2 sockets
Aug 12 05:19:58 dist-test-slave-srgv krb5kdc[9751](info): commencing operation
krb5kdc: starting...
W20260812 05:19:59.095261 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:19:59 dist-test-slave-srgv krb5kdc[9751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:19:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:19:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:19:59.220788 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39867
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:39867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:19:59.302297 9764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:19:59.302541 9764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:19:59.302579 9764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:19:59.303725 9764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:19:59.303794 9764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:19:59.303820 9764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:19:59.303853 9764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:19:59.303881 9764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:19:59.305383 9764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:39867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:19:59.305704 9764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:19:59.305941 9764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:19:59.310873 9764 server_base.cc:1034] running on GCE node
W20260812 05:19:59.310827 9770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:59.310834 9772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:19:59.310827 9769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:19:59.311336 9764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:19:59.311638 9764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:19:59.312832 9764 hybrid_clock.cc:648] HybridClock initialized: now 1786511999312819 us; error 56 us; skew 500 ppm
Aug 12 05:19:59 dist-test-slave-srgv krb5kdc[9751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786511999, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:19:59.315389 9764 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:19:59.315848 9764 webserver.cc:460] Webserver started at http://127.18.166.254:40291/ using document root <none> and password file <none>
I20260812 05:19:59.316051 9764 fs_manager.cc:359] Metadata directory not provided
I20260812 05:19:59.316090 9764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:19:59.316179 9764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:19:59.317178 9764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "450ea62ff89f4c75a2a798bee00e2ec2"
format_stamp: "Formatted at 2026-08-12 05:19:59 on dist-test-slave-srgv"
server_key: "2ee6c58d97c5f57fceed1fdedb1c9647"
server_key_iv: "27082a7b249e4c42c6486c7c2562cc1a"
server_key_version: "encryptionkey@0"
I20260812 05:19:59.317564 9764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "450ea62ff89f4c75a2a798bee00e2ec2"
format_stamp: "Formatted at 2026-08-12 05:19:59 on dist-test-slave-srgv"
server_key: "2ee6c58d97c5f57fceed1fdedb1c9647"
server_key_iv: "27082a7b249e4c42c6486c7c2562cc1a"
server_key_version: "encryptionkey@0"
I20260812 05:19:59.318856 9764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:19:59.319551 9780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:19:59.319734 9764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:19:59.319825 9764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "450ea62ff89f4c75a2a798bee00e2ec2"
format_stamp: "Formatted at 2026-08-12 05:19:59 on dist-test-slave-srgv"
server_key: "2ee6c58d97c5f57fceed1fdedb1c9647"
server_key_iv: "27082a7b249e4c42c6486c7c2562cc1a"
server_key_version: "encryptionkey@0"
I20260812 05:19:59.319881 9764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:19:59.323118 9764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:19:59.325487 9764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:19:59.325609 9764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:19:59.329517 9764 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:39867
I20260812 05:19:59.329584 9836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:39867 every 8 connection(s)
I20260812 05:19:59.329896 9764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:19:59.330889 9837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:19:59.333041 9837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2: Bootstrap starting.
I20260812 05:19:59.333636 9837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2: Neither blocks nor log segments found. Creating new log.
I20260812 05:19:59.333874 9837 log.cc:824] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2: Log is configured to *not* fsync() on all Append() calls
I20260812 05:19:59.334525 9837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2: No bootstrap required, opened a new log
I20260812 05:19:59.336016 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9764
I20260812 05:19:59.336145 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:19:59.336375 19099 external_mini_cluster.cc:1428] Setting key 04ccefa7bdefdf55e4c735f4f136bc6d
I20260812 05:19:59.336292 9837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39867 } }
I20260812 05:19:59.336458 9837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:19:59.336505 9837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 450ea62ff89f4c75a2a798bee00e2ec2, State: Initialized, Role: FOLLOWER
I20260812 05:19:59.336653 9837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39867 } }
I20260812 05:19:59.336720 9837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:19:59.336772 9837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:19:59.336826 9837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:19:59.338011 9837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39867 } }
I20260812 05:19:59.338133 9837 leader_election.cc:302] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 450ea62ff89f4c75a2a798bee00e2ec2; no voters:
I20260812 05:19:59.338311 9837 leader_election.cc:288] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:19:59.338351 9842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:19:59.338501 9842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [term 1 LEADER]: Becoming Leader. State: Replica: 450ea62ff89f4c75a2a798bee00e2ec2, State: Running, Role: LEADER
I20260812 05:19:59.338573 9837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:19:59.338631 9842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39867 } }
I20260812 05:19:59.338900 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:59.337059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49981 (local address 127.18.166.254:39867)
0812 05:19:59.337224 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:59.337233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:59.337265 (+ 32us) server_negotiation.cc:407] Connection header received
0812 05:19:59.337391 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:59.337400 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:59.337456 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:59.337645 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:59.338314 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:59.338318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:59.338318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:59.338323 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:59.338374 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:59.338392 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:59.338502 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:59.338679 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 05:19:59.338728 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260812 05:19:59.339002 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 450ea62ff89f4c75a2a798bee00e2ec2. Latest consensus state: current_term: 1 leader_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39867 } } }
I20260812 05:19:59.339094 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:59.339069 9843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "450ea62ff89f4c75a2a798bee00e2ec2" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39867 } } }
I20260812 05:19:59.339241 9843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2 [sys.catalog]: This master's current role is: LEADER
I20260812 05:19:59.339857 9850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:19:59.340117 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:19:59.340327 9850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:19:59.342610 9850 catalog_manager.cc:1347] Generated new cluster ID: a796ca3519c8469a812ceb1f6ad73a5a
I20260812 05:19:59.342670 9850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:19:59.343199 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:19:59.342025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40507 (local address 127.18.166.254:39867)
0812 05:19:59.342058 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 05:19:59.342059 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:19:59.342223 (+ 164us) server_negotiation.cc:407] Connection header received
0812 05:19:59.342281 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:19:59.342281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:19:59.342286 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:19:59.342359 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:19:59.342972 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:19:59.342974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:19:59.342974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:19:59.342979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:19:59.342992 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:19:59.342996 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:19:59.343053 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 05:19:59.343107 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 05:19:59.343120 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 05:19:59.352583 9850 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:19:59.352741 9850 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:19:59.359635 9850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 450ea62ff89f4c75a2a798bee00e2ec2: Generated new TSK 0
I20260812 05:19:59.359819 9850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:19:59.368728 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9764
2026-08-12T05:19:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:19:59 dist-test-slave-srgv krb5kdc[9869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:19:59 dist-test-slave-srgv krb5kdc[9869](info): set up 2 sockets
Aug 12 05:19:59 dist-test-slave-srgv krb5kdc[9869](info): commencing operation
krb5kdc: starting...
W20260812 05:20:00.443436 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:20:00 dist-test-slave-srgv krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:20:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:20:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:20:00.573639 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41509
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:41509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:20:00.656927 9883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:00.657162 9883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:00.657187 9883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:00.658407 9883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:00.658458 9883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:00.658473 9883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:00.658488 9883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:00.658547 9883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:00.660068 9883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:41509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:41509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:00.660351 9883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:00.660590 9883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:00.665694 9889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:00.665750 9890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:00.665728 9883 server_base.cc:1034] running on GCE node
W20260812 05:20:00.665694 9892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:00.666237 9883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:00.666553 9883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:00.667707 9883 hybrid_clock.cc:648] HybridClock initialized: now 1786512000667682 us; error 39 us; skew 500 ppm
Aug 12 05:20:00 dist-test-slave-srgv krb5kdc[9869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512000, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:00.670521 9883 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:20:00.671010 9883 webserver.cc:460] Webserver started at http://127.18.166.254:38177/ using document root <none> and password file <none>
I20260812 05:20:00.671279 9883 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:00.671355 9883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:00.671514 9883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:00.672650 9883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "8b2b6cfe5665462487b5c10429c48ff8"
format_stamp: "Formatted at 2026-08-12 05:20:00 on dist-test-slave-srgv"
server_key: "4b8d3d327c492790fc98a7277eb292c7"
server_key_iv: "f60be0b1fe622d06e48a2d85fdaf20d8"
server_key_version: "encryptionkey@0"
I20260812 05:20:00.673061 9883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "8b2b6cfe5665462487b5c10429c48ff8"
format_stamp: "Formatted at 2026-08-12 05:20:00 on dist-test-slave-srgv"
server_key: "4b8d3d327c492790fc98a7277eb292c7"
server_key_iv: "f60be0b1fe622d06e48a2d85fdaf20d8"
server_key_version: "encryptionkey@0"
I20260812 05:20:00.674610 9883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:20:00.675465 9900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:00.675678 9883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:20:00.675796 9883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "8b2b6cfe5665462487b5c10429c48ff8"
format_stamp: "Formatted at 2026-08-12 05:20:00 on dist-test-slave-srgv"
server_key: "4b8d3d327c492790fc98a7277eb292c7"
server_key_iv: "f60be0b1fe622d06e48a2d85fdaf20d8"
server_key_version: "encryptionkey@0"
I20260812 05:20:00.675884 9883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:00.696076 9883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:00.698724 9883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:00.698905 9883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:00.703028 9883 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:41509
I20260812 05:20:00.703088 9955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:41509 every 8 connection(s)
I20260812 05:20:00.703491 9883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:20:00.704473 9956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:00.706851 9956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8: Bootstrap starting.
I20260812 05:20:00.707463 9956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:00.707773 9956 log.cc:824] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:00.708523 9956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8: No bootstrap required, opened a new log
I20260812 05:20:00.709345 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 9883
I20260812 05:20:00.709484 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:20:00.709677 19099 external_mini_cluster.cc:1428] Setting key 61a7171856630dbad6b28d0d5498b8ed
I20260812 05:20:00.710867 9956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2b6cfe5665462487b5c10429c48ff8" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41509 } }
I20260812 05:20:00.711042 9956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:20:00.711084 9956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2b6cfe5665462487b5c10429c48ff8, State: Initialized, Role: FOLLOWER
I20260812 05:20:00.711258 9956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2b6cfe5665462487b5c10429c48ff8" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41509 } }
I20260812 05:20:00.711356 9956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:20:00.711392 9956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:20:00.711444 9956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:00.712459 9956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2b6cfe5665462487b5c10429c48ff8" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41509 } }
I20260812 05:20:00.712549 9956 leader_election.cc:302] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b2b6cfe5665462487b5c10429c48ff8; no voters:
I20260812 05:20:00.712781 9956 leader_election.cc:288] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:20:00.712811 9962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:20:00.712929 9962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2b6cfe5665462487b5c10429c48ff8, State: Running, Role: LEADER
I20260812 05:20:00.713013 9962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2b6cfe5665462487b5c10429c48ff8" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41509 } }
I20260812 05:20:00.713060 9956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:00.713335 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b2b6cfe5665462487b5c10429c48ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2b6cfe5665462487b5c10429c48ff8" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41509 } } }
I20260812 05:20:00.713518 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:00.713721 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2b6cfe5665462487b5c10429c48ff8. Latest consensus state: current_term: 1 leader_uuid: "8b2b6cfe5665462487b5c10429c48ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2b6cfe5665462487b5c10429c48ff8" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 41509 } } }
I20260812 05:20:00.713799 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:00.714131 9969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:20:00.714674 9969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:20:00.716434 9969 catalog_manager.cc:1347] Generated new cluster ID: 1547d04f0aa64c6ab98ddbacffdb27c3
I20260812 05:20:00.716495 9969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:20:00 dist-test-slave-srgv krb5kdc[9869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:20:00.721637 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:00.710273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43905 (local address 127.18.166.254:41509)
0812 05:20:00.710467 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:00.710478 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:00.710516 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:20:00.710847 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:00.710853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:00.710899 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:00.711014 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:00.711842 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:00.712855 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:00.713978 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:00.714660 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:00.714774 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:00.717685 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:00.717689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:00.717689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:00.717701 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:00.719701 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:00.720092 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:00.720095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:00.720095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:00.720217 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:00.720390 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:00.720391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:00.720391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:00.720676 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:00.720808 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:00.721238 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 05:20:00.721379 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260812 05:20:00.722565 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:20:00.730419 9969 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:20:00.731130 9969 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:20:00.733367 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:00.725770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49115 (local address 127.18.166.254:41509)
0812 05:20:00.725812 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:00.725814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:00.725861 (+ 47us) server_negotiation.cc:407] Connection header received
0812 05:20:00.725945 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:00.725945 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:00.725967 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:00.726013 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:00.726690 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:00.727420 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:00.728507 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:00.728830 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:00.728891 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:00.730443 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:00.730445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:00.730445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:00.730453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:00.732152 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:00.732556 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:00.732557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:00.732558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:00.732622 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:00.732827 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:00.732828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:00.732828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:00.732960 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:00.733081 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:00.733174 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 05:20:00.733273 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:20:00.734861 9969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b2b6cfe5665462487b5c10429c48ff8: Generated new TSK 0
I20260812 05:20:00.735072 9969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:20:00.741648 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 9883
2026-08-12T05:20:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:20:00 dist-test-slave-srgv krb5kdc[9989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:20:00 dist-test-slave-srgv krb5kdc[9989](info): set up 2 sockets
Aug 12 05:20:00 dist-test-slave-srgv krb5kdc[9989](info): commencing operation
krb5kdc: starting...
W20260812 05:20:01.835794 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:20:01 dist-test-slave-srgv krb5kdc[9989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:20:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:20:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:20:01.970113 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39431
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:42593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:39431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:20:02.054122 10003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:02.054402 10003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:02.054442 10003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:02.055706 10003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:02.055794 10003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:02.055817 10003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:02.055838 10003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:02.055871 10003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:02.057507 10003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:42593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:39431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:39431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:02.057821 10003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:02.058075 10003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:02.063416 10009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:02.063413 10012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:02.063413 10010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:02.063649 10003 server_base.cc:1034] running on GCE node
I20260812 05:20:02.063983 10003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:02.064343 10003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:02.065483 10003 hybrid_clock.cc:648] HybridClock initialized: now 1786512002065461 us; error 43 us; skew 500 ppm
Aug 12 05:20:02 dist-test-slave-srgv krb5kdc[9989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512002, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:02.068095 10003 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:20:02.068612 10003 webserver.cc:460] Webserver started at http://127.18.166.254:46531/ using document root <none> and password file <none>
I20260812 05:20:02.068869 10003 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:02.068917 10003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:02.069092 10003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:02.070299 10003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "8d81e60bce7e4c3889cf869f8bb9f42d"
format_stamp: "Formatted at 2026-08-12 05:20:02 on dist-test-slave-srgv"
server_key: "c5e76cff0f1adfdef04d7b3c7753de6d"
server_key_iv: "f8c356dd82b690e0a888eab839cc75f8"
server_key_version: "encryptionkey@0"
I20260812 05:20:02.070673 10003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "8d81e60bce7e4c3889cf869f8bb9f42d"
format_stamp: "Formatted at 2026-08-12 05:20:02 on dist-test-slave-srgv"
server_key: "c5e76cff0f1adfdef04d7b3c7753de6d"
server_key_iv: "f8c356dd82b690e0a888eab839cc75f8"
server_key_version: "encryptionkey@0"
I20260812 05:20:02.072074 10003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:20:02.072839 10019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:02.073027 10003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:20:02.073096 10003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "8d81e60bce7e4c3889cf869f8bb9f42d"
format_stamp: "Formatted at 2026-08-12 05:20:02 on dist-test-slave-srgv"
server_key: "c5e76cff0f1adfdef04d7b3c7753de6d"
server_key_iv: "f8c356dd82b690e0a888eab839cc75f8"
server_key_version: "encryptionkey@0"
I20260812 05:20:02.073195 10003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:02.092638 10003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:02.094892 10003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:02.095059 10003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:02.099680 10003 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:39431
I20260812 05:20:02.099717 10078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:39431 every 8 connection(s)
I20260812 05:20:02.100113 10003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:20:02.101147 10079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:02.103509 10079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d: Bootstrap starting.
I20260812 05:20:02.104111 10079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:02.104358 10079 log.cc:824] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:02.105124 10079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d: No bootstrap required, opened a new log
I20260812 05:20:02.105926 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10003
I20260812 05:20:02.106088 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:20:02.106299 19099 external_mini_cluster.cc:1428] Setting key efcd46d52530f5f4da6751165d79f447
I20260812 05:20:02.107534 10079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39431 } }
I20260812 05:20:02.107735 10079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:20:02.107780 10079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d81e60bce7e4c3889cf869f8bb9f42d, State: Initialized, Role: FOLLOWER
I20260812 05:20:02.107956 10079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39431 } }
I20260812 05:20:02.108069 10079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:20:02.108104 10079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:20:02.108147 10079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:02.109032 10079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39431 } }
I20260812 05:20:02.109149 10079 leader_election.cc:302] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d81e60bce7e4c3889cf869f8bb9f42d; no voters:
I20260812 05:20:02.109310 10079 leader_election.cc:288] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:20:02.109391 10085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:20:02.109589 10079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:02.109598 10085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [term 1 LEADER]: Becoming Leader. State: Replica: 8d81e60bce7e4c3889cf869f8bb9f42d, State: Running, Role: LEADER
I20260812 05:20:02.109800 10085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39431 } }
I20260812 05:20:02.111112 10086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d81e60bce7e4c3889cf869f8bb9f42d. Latest consensus state: current_term: 1 leader_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39431 } } }
I20260812 05:20:02.111404 10086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:02.111373 10087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d81e60bce7e4c3889cf869f8bb9f42d" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 39431 } } }
I20260812 05:20:02.111450 10087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d [sys.catalog]: This master's current role is: LEADER
W20260812 05:20:02.111903 10100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:20:02.111964 10100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:20:02.112365 10101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:20:02.112582 10101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:20:02.113258 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:02.106891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58051 (local address 127.18.166.254:39431)
0812 05:20:02.107119 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:02.107129 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:02.107176 (+ 47us) server_negotiation.cc:407] Connection header received
0812 05:20:02.107487 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:02.107496 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:02.107565 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:02.107729 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:02.108052 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:02.109812 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:02.110973 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:02.111635 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:02.111742 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:02.112560 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:02.112564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:02.112564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:20:02.112794 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:02.112826 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:20:02.112838 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:02.112913 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:02.112989 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 05:20:02.113058 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260812 05:20:02.114482 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:20:02.117925 10101 catalog_manager.cc:1347] Generated new cluster ID: d59ebe23872c4e8e9457ab39cea927a6
I20260812 05:20:02.118006 10101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:20:02.121676 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:02.116072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53197 (local address 127.18.166.254:39431)
0812 05:20:02.116121 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:02.116123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:02.118074 (+ 1951us) server_negotiation.cc:407] Connection header received
0812 05:20:02.118194 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:02.118196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:02.118209 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:02.118276 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:02.118441 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:02.119159 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:02.120239 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:02.120544 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:02.120614 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:02.121398 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:02.121400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:02.121400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:20:02.121423 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:02.121441 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:20:02.121446 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:02.121498 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:02.121561 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 05:20:02.121592 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:20:02.129007 10101 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:20:02.129694 10101 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:20:02.135193 10101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d81e60bce7e4c3889cf869f8bb9f42d: Generated new TSK 0
I20260812 05:20:02.135437 10101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:20:02.144863 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10003
2026-08-12T05:20:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:20:02 dist-test-slave-srgv krb5kdc[10114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:20:02 dist-test-slave-srgv krb5kdc[10114](info): set up 2 sockets
Aug 12 05:20:02 dist-test-slave-srgv krb5kdc[10114](info): commencing operation
krb5kdc: starting...
W20260812 05:20:03.215737 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:20:03 dist-test-slave-srgv krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:20:03.349048 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34213
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:34213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:20:03.434556 10127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:03.434806 10127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:03.434845 10127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:03.435995 10127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:03.436076 10127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:03.436105 10127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:03.436151 10127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:03.436182 10127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:03.437767 10127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:34213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:34213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:03.438038 10127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:03.438297 10127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:03.443285 10133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:03.443295 10132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:03.443295 10135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:03.443542 10127 server_base.cc:1034] running on GCE node
I20260812 05:20:03.443792 10127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:03.444105 10127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:03.445288 10127 hybrid_clock.cc:648] HybridClock initialized: now 1786512003445271 us; error 46 us; skew 500 ppm
Aug 12 05:20:03 dist-test-slave-srgv krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512003, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:03.447988 10127 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:20:03.448499 10127 webserver.cc:460] Webserver started at http://127.18.166.254:37459/ using document root <none> and password file <none>
I20260812 05:20:03.448751 10127 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:03.448820 10127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:03.448971 10127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:03.450109 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5"
format_stamp: "Formatted at 2026-08-12 05:20:03 on dist-test-slave-srgv"
server_key: "f5153b95caa581b9681b9ceb65dbc426"
server_key_iv: "235d91883d633811e68162dbf5199176"
server_key_version: "encryptionkey@0"
I20260812 05:20:03.450486 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5"
format_stamp: "Formatted at 2026-08-12 05:20:03 on dist-test-slave-srgv"
server_key: "f5153b95caa581b9681b9ceb65dbc426"
server_key_iv: "235d91883d633811e68162dbf5199176"
server_key_version: "encryptionkey@0"
I20260812 05:20:03.451865 10127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:20:03.452684 10143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:03.452863 10127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:20:03.452970 10127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5"
format_stamp: "Formatted at 2026-08-12 05:20:03 on dist-test-slave-srgv"
server_key: "f5153b95caa581b9681b9ceb65dbc426"
server_key_iv: "235d91883d633811e68162dbf5199176"
server_key_version: "encryptionkey@0"
I20260812 05:20:03.453045 10127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:03.456475 10127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:03.458989 10127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:03.459158 10127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:03.463495 10127 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:34213
I20260812 05:20:03.463526 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:34213 every 8 connection(s)
I20260812 05:20:03.463896 10127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:20:03.464775 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10127
I20260812 05:20:03.464905 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:20:03.465065 10201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:03.465158 19099 external_mini_cluster.cc:1428] Setting key df3f11bfe08fab934231b6c14ff1ee0c
I20260812 05:20:03.467784 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:03.465708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33381 (local address 127.18.166.254:34213)
0812 05:20:03.465888 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:03.465895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:03.465919 (+ 24us) server_negotiation.cc:407] Connection header received
0812 05:20:03.466386 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:03.466392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:03.466418 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:03.466543 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:03.467263 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:03.467266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:03.467266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:20:03.467270 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:03.467303 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:20:03.467313 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:03.467370 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:03.467500 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 05:20:03.467563 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260812 05:20:03.468804 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:20:03.469348 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5: Bootstrap starting.
I20260812 05:20:03.470086 10201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:03.470474 10201 log.cc:824] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:03.471387 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5: No bootstrap required, opened a new log
I20260812 05:20:03.471540 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:03.470273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39141 (local address 127.18.166.254:34213)
0812 05:20:03.470324 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:03.470326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:03.470456 (+ 130us) server_negotiation.cc:407] Connection header received
0812 05:20:03.470524 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:03.470525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:03.470529 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:03.470605 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:03.471292 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:03.471293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:03.471293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:20:03.471297 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:03.471312 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:20:03.471316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:03.471363 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:03.471458 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 05:20:03.471471 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 05:20:03.473383 10201 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34213 } }
I20260812 05:20:03.473589 10201 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:20:03.473618 10201 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ffe7cf5ae44f65a3e7e49d5c5ff1d5, State: Initialized, Role: FOLLOWER
I20260812 05:20:03.473743 10201 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34213 } }
I20260812 05:20:03.473827 10201 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:20:03.473850 10201 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:20:03.473878 10201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:03.474397 10201 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34213 } }
I20260812 05:20:03.474480 10201 leader_election.cc:302] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0ffe7cf5ae44f65a3e7e49d5c5ff1d5; no voters:
I20260812 05:20:03.474613 10201 leader_election.cc:288] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:20:03.474690 10211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:20:03.474839 10211 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [term 1 LEADER]: Becoming Leader. State: Replica: a0ffe7cf5ae44f65a3e7e49d5c5ff1d5, State: Running, Role: LEADER
I20260812 05:20:03.474915 10201 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:03.474926 10211 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34213 } }
I20260812 05:20:03.475322 10213 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0ffe7cf5ae44f65a3e7e49d5c5ff1d5. Latest consensus state: current_term: 1 leader_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34213 } } }
I20260812 05:20:03.475328 10212 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0ffe7cf5ae44f65a3e7e49d5c5ff1d5" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 34213 } } }
I20260812 05:20:03.475523 10212 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:03.475793 10213 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:03.475872 10219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:20:03.476385 10219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:20:03.478181 10219 catalog_manager.cc:1347] Generated new cluster ID: dc83fc787f8e4eb19a5cfdf1883f43b0
I20260812 05:20:03.478235 10219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:20:03.489148 10219 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:20:03.489351 10219 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:20:03.491961 10219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0ffe7cf5ae44f65a3e7e49d5c5ff1d5: Generated new TSK 0
I20260812 05:20:03.492141 10219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:20:03.503134 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10127
2026-08-12T05:20:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1358 ms)
[----------] 18 tests from AuthTokenIssuingTest (24629 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:20:03 dist-test-slave-srgv krb5kdc[10233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:20:03 dist-test-slave-srgv krb5kdc[10233](info): set up 2 sockets
Aug 12 05:20:03 dist-test-slave-srgv krb5kdc[10233](info): commencing operation
krb5kdc: starting...
W20260812 05:20:04.571864 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:20:04 dist-test-slave-srgv krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:20:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:20:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:20:04.700876 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:33215
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:33215,127.18.166.253:44657,127.18.166.252:44693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:20:04.784976 10246 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:20:04.785283 10246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:04.785318 10246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:04.785365 10246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:04.786633 10246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:04.786710 10246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:04.786734 10246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:04.786756 10246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:04.786785 10246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:04.788326 10246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:33215,127.18.166.253:44657,127.18.166.252:44693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:33215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:04.788631 10246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:04.788872 10246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:04.793983 10253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:04.793983 10256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:04.794340 10246 server_base.cc:1034] running on GCE node
W20260812 05:20:04.794025 10254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:04.794643 10246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:04.794981 10246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:04.796185 10246 hybrid_clock.cc:648] HybridClock initialized: now 1786512004796168 us; error 52 us; skew 500 ppm
Aug 12 05:20:04 dist-test-slave-srgv krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:04.804725 10246 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:20:04.805565 10246 webserver.cc:460] Webserver started at http://127.18.166.254:43959/ using document root <none> and password file <none>
I20260812 05:20:04.805938 10246 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:04.806061 10246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:04.806206 10246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:04.807374 10246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb"
format_stamp: "Formatted at 2026-08-12 05:20:04 on dist-test-slave-srgv"
server_key: "9767696ee9dff19db03a90b816aff0f1"
server_key_iv: "d7112cc5cd6e8168b78acb2dc2f8eb42"
server_key_version: "encryptionkey@0"
I20260812 05:20:04.807770 10246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb"
format_stamp: "Formatted at 2026-08-12 05:20:04 on dist-test-slave-srgv"
server_key: "9767696ee9dff19db03a90b816aff0f1"
server_key_iv: "d7112cc5cd6e8168b78acb2dc2f8eb42"
server_key_version: "encryptionkey@0"
I20260812 05:20:04.809402 10246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:20:04.810329 10263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:04.810565 10246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:20:04.810683 10246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb"
format_stamp: "Formatted at 2026-08-12 05:20:04 on dist-test-slave-srgv"
server_key: "9767696ee9dff19db03a90b816aff0f1"
server_key_iv: "d7112cc5cd6e8168b78acb2dc2f8eb42"
server_key_version: "encryptionkey@0"
I20260812 05:20:04.810781 10246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:04.820320 10246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:04.822891 10246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:04.823081 10246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:04.827678 10246 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:33215
I20260812 05:20:04.827731 10319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:33215 every 8 connection(s)
I20260812 05:20:04.828136 10246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:20:04.829031 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:04.830749 10320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:04.836635 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10246
I20260812 05:20:04.836743 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:20:04.836949 19099 external_mini_cluster.cc:1428] Setting key bd4d4344c3f5dbb79a10ba923c85dadb
Aug 12 05:20:04 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:20:04.841266 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.831671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.18.166.254:33215)
0812 05:20:04.831832 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:04.831836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:04.831866 (+ 30us) server_negotiation.cc:407] Connection header received
0812 05:20:04.832331 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:04.832333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:04.832359 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:04.832442 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:04.833020 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.833968 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.834815 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.835129 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.835211 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:04.837712 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:04.837721 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:04.837722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:04.837729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:04.839556 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:04.840060 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:04.840063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:04.840063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:04.840149 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:04.840412 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:04.840412 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:04.840412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:04.840630 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:04.840693 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:04.840931 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 05:20:04.841035 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260812 05:20:04.841296 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.831483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:33215 (local address 127.0.0.1:52886)
0812 05:20:04.831667 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.831703 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:04.832164 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:04.832472 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:04.832474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:04.832743 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:04.832902 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:04.832906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.834057 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.834058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:04.834735 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:04.834736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.835208 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.835209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:04.835277 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:04.835849 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:04.835858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:04.837624 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:04.839670 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:04.839677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:04.839680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:04.839967 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:04.840223 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:04.840224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:04.840224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:04.840344 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:04.840700 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:04.840707 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:04.840785 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 05:20:04.840898 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 05:20:04.841151 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":65,"thread_start_us":93,"threads_started":1}
I20260812 05:20:04.842334 10320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:04.842725 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.842590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38126)
0812 05:20:04.842643 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.842696 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 05:20:04.842905 10266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
I20260812 05:20:04.843393 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.843307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38138)
0812 05:20:04.843352 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.843377 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:20:04.843521 10320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:44657: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.166.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab.
Entry for principal kudu/127.18.166.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab.
I20260812 05:20:04.865180 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:44657
--webserver_interface=127.18.166.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab
--principal=kudu/127.18.166.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:33215,127.18.166.253:44657,127.18.166.252:44693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:20:04.892764 10320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } attempt: 1
I20260812 05:20:04.893319 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.893168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38154)
0812 05:20:04.893250 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.893289 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 05:20:04.893759 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.893664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38156)
0812 05:20:04.893714 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.893741 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 05:20:04.893850 10320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:44657: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
W20260812 05:20:04.948683 10327 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:20:04.948904 10327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:04.948922 10327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:04.948978 10327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:04.950187 10327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:04.950229 10327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:04.950244 10327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:04.950294 10327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:04.950321 10327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:04.951809 10327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:33215,127.18.166.253:44657,127.18.166.252:44693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab
--principal=kudu/127.18.166.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:44657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.166.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:04.952075 10327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:04.952315 10327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:04.957224 10333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:04.957266 10334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:04.957239 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:04.957484 10327 server_base.cc:1034] running on GCE node
I20260812 05:20:04.957798 10327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:04.958122 10327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:04.959298 10327 hybrid_clock.cc:648] HybridClock initialized: now 1786512004959287 us; error 28 us; skew 500 ppm
Aug 12 05:20:04 dist-test-slave-srgv krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:04.961786 10327 init.cc:377] Logged in from keytab as kudu/127.18.166.253@KRBTEST.COM (short username kudu)
I20260812 05:20:04.962009 10320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } attempt: 2
I20260812 05:20:04.962322 10327 webserver.cc:460] Webserver started at http://127.18.166.253:37827/ using document root <none> and password file <none>
I20260812 05:20:04.962467 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.962309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38158)
0812 05:20:04.962375 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.962433 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:20:04.962620 10327 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:04.962703 10327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:04.962832 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.962725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38162)
0812 05:20:04.962771 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.962804 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 05:20:04.962859 10327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
W20260812 05:20:04.962944 10320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:44657: Network error: Client connection negotiation failed: client connection to 127.18.166.253:44657: connect: Connection refused (error 111)
I20260812 05:20:04.963976 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data/instance:
uuid: "2c09b2b6df694e169195cd995e71a092"
format_stamp: "Formatted at 2026-08-12 05:20:04 on dist-test-slave-srgv"
server_key: "c0fbc089b250fa4ef0244bdf6a22ea1b"
server_key_iv: "c0528d83b7c20a4efa046332bc9fc0b9"
server_key_version: "encryptionkey@0"
I20260812 05:20:04.964355 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal/instance:
uuid: "2c09b2b6df694e169195cd995e71a092"
format_stamp: "Formatted at 2026-08-12 05:20:04 on dist-test-slave-srgv"
server_key: "c0fbc089b250fa4ef0244bdf6a22ea1b"
server_key_iv: "c0528d83b7c20a4efa046332bc9fc0b9"
server_key_version: "encryptionkey@0"
I20260812 05:20:04.965797 10327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:20:04.966526 10343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:04.966704 10327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:20:04.966787 10327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal
uuid: "2c09b2b6df694e169195cd995e71a092"
format_stamp: "Formatted at 2026-08-12 05:20:04 on dist-test-slave-srgv"
server_key: "c0fbc089b250fa4ef0244bdf6a22ea1b"
server_key_iv: "c0528d83b7c20a4efa046332bc9fc0b9"
server_key_version: "encryptionkey@0"
I20260812 05:20:04.966845 10327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:04.980109 10327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:04.982224 10327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:04.982350 10327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:04.986447 10327 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.253:44657
I20260812 05:20:04.986485 10398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.253:44657 every 8 connection(s)
I20260812 05:20:04.986876 10327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
I20260812 05:20:04.987807 10399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:04.989473 10399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:04.990562 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10327
I20260812 05:20:04.990670 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-1/wal/instance
I20260812 05:20:04.990875 19099 external_mini_cluster.cc:1428] Setting key ead1eaa3987ad064da0e61f54008c031
Aug 12 05:20:04 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.166.252@KRBTEST.COM; defaulting to no policy
I20260812 05:20:04.999799 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.990251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.18.166.254:33215)
0812 05:20:04.990318 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:04.990320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:04.990545 (+ 225us) server_negotiation.cc:407] Connection header received
0812 05:20:04.991262 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:04.991263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:04.991276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:04.991332 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:04.992180 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.992673 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.994218 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.994428 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.994484 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:04.996776 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:04.996778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:04.996778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:04.996793 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:04.998299 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:04.998745 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:04.998747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:04.998747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:04.998812 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:04.999110 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:04.999111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:04.999112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:04.999241 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:04.999315 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:04.999602 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 05:20:04.999693 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:20:05.000025 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:04.990203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:33215 (local address 127.0.0.1:52898)
0812 05:20:04.990387 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:04.990439 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:04.991156 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:04.991401 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:04.991406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:04.991808 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:04.992093 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:04.992099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.992773 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.992774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:04.994134 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:04.994137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:04.994517 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:04.994518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:04.994670 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:04.995256 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:04.995265 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:04.996665 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:04.998406 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:04.998415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:04.998418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:04.998657 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:04.998887 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:04.998888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:04.998888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:04.999028 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:04.999321 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:04.999329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:04.999470 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 05:20:04.999581 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 05:20:04.999746 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260812 05:20:05.000609 10399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.252@KRBTEST.COM" created.
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:05.009516 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.000910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38174)
0812 05:20:05.000968 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.000981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.001031 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.001391 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.001392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.001557 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.001605 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.001606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.003286 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.003287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.003780 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.003781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.004475 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.004475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.004529 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.004953 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.004957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.006189 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.008386 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.008388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.008388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.008552 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.008815 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.008816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.008816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.008871 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.009216 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.009217 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.009257 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.009305 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.009390 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 05:20:05.009644 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.001023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.18.166.253:44657)
0812 05:20:05.001255 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.001259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.001275 (+ 16us) server_negotiation.cc:407] Connection header received
0812 05:20:05.001292 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.001293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.001304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.001392 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.002092 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.003178 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.003863 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.004388 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.004478 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.006302 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.006311 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.006311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.006319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.008304 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.008639 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.008643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.008643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.008713 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.008936 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.008937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.008937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.009141 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.009230 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.009345 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.009497 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":205,"thread_start_us":164,"threads_started":1}
I20260812 05:20:05.010288 10399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:05.010632 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.010535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:42942)
0812 05:20:05.010569 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.010613 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 05:20:05.010793 10346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
I20260812 05:20:05.011183 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.011118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:42948)
0812 05:20:05.011144 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.011167 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 05:20:05.011325 10399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:44693: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab.
Entry for principal kudu/127.18.166.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab.
I20260812 05:20:05.020149 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:44693
--webserver_interface=127.18.166.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab
--principal=kudu/127.18.166.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:33215,127.18.166.253:44657,127.18.166.252:44693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:20:05.054149 10320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } attempt: 3
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:05.060549 10399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } attempt: 1
I20260812 05:20:05.060935 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.060816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:42954)
0812 05:20:05.060870 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.060903 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 05:20:05.061436 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.061292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:42968)
0812 05:20:05.061354 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.061388 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 05:20:05.061568 10399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:44693: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
I20260812 05:20:05.062024 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.054708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.18.166.253:44657)
0812 05:20:05.054795 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.054797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.054810 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:20:05.054844 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.054845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.054857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.054930 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.055296 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.055790 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.056518 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.056748 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.056824 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.059047 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.059049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.059050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.059058 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.060611 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.061057 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.061058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.061059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.061122 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.061388 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.061389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.061390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.061606 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.061675 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.061846 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.061931 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 05:20:05.062026 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.054601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38182)
0812 05:20:05.054678 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.054699 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.054783 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.054937 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.054938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.055126 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.055197 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.055200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.055878 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.055880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.056425 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.056427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.056835 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.056836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.056919 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.057464 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.057469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.058936 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.060725 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.060726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.060726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.060957 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.061229 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.061230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.061231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.061328 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.061682 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.061683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.061731 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.061819 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.061929 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 05:20:05.062423 10320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:05.062731 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.062650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:42984)
0812 05:20:05.062682 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.062707 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 05:20:05.063020 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.062936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:42994)
0812 05:20:05.062978 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.063001 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 05:20:05.063117 10320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:44693: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
I20260812 05:20:05.094257 10320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } attempt: 1
I20260812 05:20:05.094760 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.094613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43008)
0812 05:20:05.094679 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.094717 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:20:05.095629 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.095512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43010)
0812 05:20:05.095557 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.095595 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 05:20:05.095714 10320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:44693: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
W20260812 05:20:05.106060 10406 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:20:05.106329 10406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:05.106395 10406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:05.106442 10406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:05.107671 10406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:05.107745 10406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:05.107772 10406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:05.107805 10406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:05.107834 10406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:05.109587 10406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:33215,127.18.166.253:44657,127.18.166.252:44693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab
--principal=kudu/127.18.166.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:44693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.166.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:05.109880 10406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:05.110116 10406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:05.114971 10411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:05.115191 10412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:05.115171 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:05.115396 10406 server_base.cc:1034] running on GCE node
I20260812 05:20:05.115674 10406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:05.115975 10406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:05.117162 10406 hybrid_clock.cc:648] HybridClock initialized: now 1786512005117119 us; error 62 us; skew 500 ppm
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:05.119822 10406 init.cc:377] Logged in from keytab as kudu/127.18.166.252@KRBTEST.COM (short username kudu)
I20260812 05:20:05.120360 10406 webserver.cc:460] Webserver started at http://127.18.166.252:36979/ using document root <none> and password file <none>
I20260812 05:20:05.120617 10406 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:05.120690 10406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:05.120849 10406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:05.122025 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data/instance:
uuid: "857d9ccccac642fe9982e3bd92a5277d"
format_stamp: "Formatted at 2026-08-12 05:20:05 on dist-test-slave-srgv"
server_key: "0db0c7cfad228f12d039d1e6498ac819"
server_key_iv: "45491013593cffe5012bd52f1a4f9915"
server_key_version: "encryptionkey@0"
I20260812 05:20:05.122401 10406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal/instance:
uuid: "857d9ccccac642fe9982e3bd92a5277d"
format_stamp: "Formatted at 2026-08-12 05:20:05 on dist-test-slave-srgv"
server_key: "0db0c7cfad228f12d039d1e6498ac819"
server_key_iv: "45491013593cffe5012bd52f1a4f9915"
server_key_version: "encryptionkey@0"
I20260812 05:20:05.123824 10406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:20:05.124598 10421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:05.124778 10406 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:20:05.124881 10406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal
uuid: "857d9ccccac642fe9982e3bd92a5277d"
format_stamp: "Formatted at 2026-08-12 05:20:05 on dist-test-slave-srgv"
server_key: "0db0c7cfad228f12d039d1e6498ac819"
server_key_iv: "45491013593cffe5012bd52f1a4f9915"
server_key_version: "encryptionkey@0"
I20260812 05:20:05.124962 10406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:05.129724 10399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } attempt: 2
I20260812 05:20:05.130172 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.130024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43026)
0812 05:20:05.130090 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.130134 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:20:05.130525 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.130441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43036)
0812 05:20:05.130483 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.130508 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 05:20:05.130643 10399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:44693: Network error: Client connection negotiation failed: client connection to 127.18.166.252:44693: connect: Connection refused (error 111)
I20260812 05:20:05.135102 10406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:05.136992 10406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:05.137102 10406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:05.141352 10406 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.252:44693
I20260812 05:20:05.141387 10476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.252:44693 every 8 connection(s)
I20260812 05:20:05.141825 10406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
I20260812 05:20:05.142759 10477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:05.144882 10477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:05.145530 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10406
I20260812 05:20:05.145653 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786511731160932-19099-0/minicluster-data/master-2/wal/instance
I20260812 05:20:05.145941 19099 external_mini_cluster.cc:1428] Setting key 279aede58708a538fa13fbcc63a0e233
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:20:05.155187 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.145729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.18.166.254:33215)
0812 05:20:05.145782 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.145784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.146482 (+ 698us) server_negotiation.cc:407] Connection header received
0812 05:20:05.146918 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.146920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.146932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.146997 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.147549 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.148036 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.149095 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.149435 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.149508 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.152041 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.152042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.152043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.152051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.153779 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.154205 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.154206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.154207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.154272 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.154530 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.154531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.154531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.154658 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.154736 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.155009 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.155092 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:20:05.155495 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.145732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:33215 (local address 127.0.0.1:52912)
0812 05:20:05.145985 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.146036 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.146790 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.147026 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.147028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.147300 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.147451 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.147455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.148115 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.148116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.149014 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.149016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.149513 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.149514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.149644 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.150297 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.150307 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.151947 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.153863 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.153871 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.153873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.154128 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.154351 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.154352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.154352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.154460 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.154733 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.154741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.154878 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.154988 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.155144 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
I20260812 05:20:05.155808 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.146384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.18.166.252:44693)
0812 05:20:05.146556 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.146563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.146602 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:20:05.146790 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.146798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.146824 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.146909 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.147600 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.148568 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.149465 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.149898 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.149969 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.152579 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.152586 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.152586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.152592 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.154267 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.154655 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.154658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.154659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.154718 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.154922 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.154923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.154923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.155066 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.155228 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.155480 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.155558 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260812 05:20:05.156076 10477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:20:05.163141 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.156401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:44657 (local address 127.0.0.1:38190)
0812 05:20:05.156445 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.156454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.156487 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.156648 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.156649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.156799 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.156855 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.156857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.157533 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.157534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.158019 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.158020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.158414 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.158415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.158485 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.158965 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.158970 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.160297 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.162059 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.162060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.162060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.162250 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.162492 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.162493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.162493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.162558 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.162800 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.162801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.162849 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.162932 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.163038 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 05:20:05.163146 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.156475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.18.166.253:44657)
0812 05:20:05.156520 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.156522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.156531 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:20:05.156557 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.156558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.156569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.156627 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.156951 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.157469 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.158106 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.158341 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.158418 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.160401 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.160402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.160403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.160410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.161964 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.162353 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.162355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.162355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.162423 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.162608 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.162609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.162610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.162729 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.162808 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.162975 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.163049 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:20:05.163489 10477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:05.164959 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.156708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52918 (local address 127.18.166.254:33215)
0812 05:20:05.156770 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.156772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.156779 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:20:05.156796 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.156797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.156805 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.156878 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.157335 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.157832 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.158844 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.159138 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.159209 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.161646 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.161648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.161648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.161654 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.163092 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.163642 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.163643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.163643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.163701 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.163941 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.163942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.163942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.164106 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.164745 (+ 639us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.164780 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.164849 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:05.170753 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.163767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43056)
0812 05:20:05.163819 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.163829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.163867 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.164008 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.164008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.164180 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.164216 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.164218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.164906 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.164907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.165716 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.165719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.166216 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.166217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.166258 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.166671 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.166676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.167792 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.169465 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.169466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.169467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.169717 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.169919 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.169920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.169920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.169985 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.170303 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.170304 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.170376 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.170466 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.170607 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 05:20:05.170850 10320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } attempt: 2
I20260812 05:20:05.171130 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.163818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.18.166.252:44693)
0812 05:20:05.163884 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.163886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.163906 (+ 20us) server_negotiation.cc:407] Connection header received
0812 05:20:05.163928 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.163929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.163940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.163994 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.164292 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.164826 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.165799 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.166132 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.166204 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.167875 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.167877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.167877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.167884 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.169357 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.169801 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.169802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.169802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.169865 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.170051 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.170052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.170053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.170189 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.170273 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.170509 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.170692 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":43}
I20260812 05:20:05.173125 10477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: Bootstrap starting.
I20260812 05:20:05.173873 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.165694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.18.166.253:44657)
0812 05:20:05.165743 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.165745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.165819 (+ 74us) server_negotiation.cc:407] Connection header received
0812 05:20:05.165884 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.165884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.165892 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.165933 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.166500 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.166942 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.167648 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.167893 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.167965 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.170831 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.170832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.170833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.170841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.172555 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.172986 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.172988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.172988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.173055 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.173289 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.173290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.173290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.173471 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.173664 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.173703 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.173782 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:20:05.174332 10477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:05.174646 10477 log.cc:824] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:05.175511 10477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: No bootstrap required, opened a new log
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:20:05.175833 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:20:05.177731 10477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.177850 10477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857d9ccccac642fe9982e3bd92a5277d, State: Initialized, Role: FOLLOWER
I20260812 05:20:05.177907 10477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.177094 10291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52918
I20260812 05:20:05.178174 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.178273 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:05.178227 10477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:05.179405 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.171190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.18.166.252:44693)
0812 05:20:05.171249 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.171251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.171259 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:20:05.171288 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.171288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.171296 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.171915 (+ 619us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.171950 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.172669 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.173624 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.173934 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.174277 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.176323 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.176325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.176325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.176333 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.178058 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.178478 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.178479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.178479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.178535 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.178785 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.178786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.178786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.178960 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.179030 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.179231 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.179310 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:20:05.179461 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.171129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43060)
0812 05:20:05.171187 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.171197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.171237 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.171423 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.171424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.171591 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.171642 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.171644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.172758 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.172760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.173536 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.173539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.174035 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.174036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.174101 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.174676 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.174682 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.176201 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.178165 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.178165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.178166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.178388 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.178622 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.178623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.178624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.178702 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.179037 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.179038 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.179110 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.179206 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.179319 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 05:20:05.180583 10494 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:20:05.180698 10494 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:20:05.181138 10320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb: Bootstrap starting.
I20260812 05:20:05.181932 10320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:05.182273 10320 log.cc:824] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:05.183319 10320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb: No bootstrap required, opened a new log
I20260812 05:20:05.180783 10406 master_runner.cc:186] Error getting master registration for 127.18.166.254:33215: OK, instance_id { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" instance_seqno: 1786512004826399 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:20:05.183847 10406 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:20:05.184746 10320 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.184891 10320 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c4ffb4e9004c8d9a283f1e095d8ccb, State: Initialized, Role: FOLLOWER
I20260812 05:20:05.184985 10320 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.185303 10320 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:05.185329 10291 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 05:20:05.185385 10291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:05.186228 10291 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.186522 10291 leader_election.cc:288] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [CANDIDATE]: Term 1 election: Requested vote from peers 2c09b2b6df694e169195cd995e71a092 (127.18.166.253:44657), 857d9ccccac642fe9982e3bd92a5277d (127.18.166.252:44693)
I20260812 05:20:05.186859 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "857d9ccccac642fe9982e3bd92a5277d"
I20260812 05:20:05.187031 10449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:05.187983 10449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c4ffb4e9004c8d9a283f1e095d8ccb in term 1.
I20260812 05:20:05.188446 10266 leader_election.cc:302] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857d9ccccac642fe9982e3bd92a5277d, e2c4ffb4e9004c8d9a283f1e095d8ccb; no voters:
I20260812 05:20:05.193528 10514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:20:05.193737 10514 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [term 1 LEADER]: Becoming Leader. State: Replica: e2c4ffb4e9004c8d9a283f1e095d8ccb, State: Running, Role: LEADER
I20260812 05:20:05.193840 10514 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.190335 10246 master_runner.cc:186] Error getting master registration for 127.18.166.253:44657: OK, instance_id { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" instance_seqno: 1786512004985174 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:20:05.194051 10246 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:20:05.194386 10514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c4ffb4e9004c8d9a283f1e095d8ccb. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.194496 10514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:05.185269 10497 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.196164 10497 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:05.198143 10520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:20:05.198762 10520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:20:05.198902 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.188492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.18.166.252:44693)
0812 05:20:05.188531 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.188533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.188540 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:20:05.188559 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.188559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.188569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.188614 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.191205 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.191899 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.192920 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.193210 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.193255 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.195187 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.195189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.195190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.195199 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.197471 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.197913 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.197914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.197915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.197975 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.198191 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.198192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.198193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.198342 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.198423 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.198671 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.198777 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:20:05.199560 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.188307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.18.166.254:33215)
0812 05:20:05.188486 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.188488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.188496 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:20:05.188520 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.188521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.188531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.188586 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.189471 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.190249 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.191368 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.191665 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.191704 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.195820 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.195821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.195822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.195829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.197995 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.198641 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.198642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.198643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.198709 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.198936 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.198936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.198937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.199063 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.199143 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.199370 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.199452 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260812 05:20:05.200666 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.188722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.18.166.253:44657)
0812 05:20:05.188772 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.188774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.188781 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:20:05.188803 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.188804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.188813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.188866 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.189605 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.190298 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.192897 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.193782 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.193836 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.195837 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.195838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.195839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.195848 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.197898 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.198641 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.198643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.198643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.198719 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.200025 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.200026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.200027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.200172 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.200258 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.200481 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.200571 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:20:05.201014 10449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [term 1 FOLLOWER]: Refusing update from remote peer e2c4ffb4e9004c8d9a283f1e095d8ccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:20:05.201511 10518 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:20:05.203078 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c4ffb4e9004c8d9a283f1e095d8ccb. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.203188 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:05.206569 10514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.206682 10514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:05.206669 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.206763 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:05.206928 10522 mvcc.cc:205] Tried to move back new op lower bound from 7317553173299376128 to 7317553173275021312. Current Snapshot: MvccSnapshot[applied={T|T < 7317553173299376128}]
I20260812 05:20:05.207119 10523 mvcc.cc:205] Tried to move back new op lower bound from 7317553173299376128 to 7317553173275021312. Current Snapshot: MvccSnapshot[applied={T|T < 7317553173299376128}]
I20260812 05:20:05.207686 10520 catalog_manager.cc:1347] Generated new cluster ID: 05d30bad004043a99dd367445b90a624
I20260812 05:20:05.207762 10520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:20:05.221352 10520 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:20:05.221799 10399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } attempt: 3
I20260812 05:20:05.222700 10520 catalog_manager.cc:1504] Loading token signing keys...
Aug 12 05:20:05 dist-test-slave-srgv krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512004, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:20:05.230562 10520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb: Generated new TSK 0
I20260812 05:20:05.230813 10520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:20:05.231532 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.222071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:44693 (local address 127.0.0.1:43084)
0812 05:20:05.222163 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:05.222174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:05.222229 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:05.222615 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:05.222616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:05.222840 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:05.222893 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.222895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.224271 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.224272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.225057 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:05.225059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.225605 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.225606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:05.225678 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.226245 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:05.226251 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:05.227921 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:05.230019 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.230020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.230020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.230262 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.230534 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:05.230535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:05.230536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:05.230626 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:05.231171 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:05.231172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:05.231225 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 05:20:05.231295 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:20:05.231419 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 05:20:05.231917 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:05.222328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.18.166.252:44693)
0812 05:20:05.222375 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.222377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.222385 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:20:05.222406 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.222407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.222417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.222465 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.223116 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.223813 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.225143 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:05.225480 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:05.225532 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:05.228029 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:05.228031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:05.228031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:05.228040 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:05.229913 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.230341 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.230342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.230342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.230431 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:05.230831 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:05.230832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:05.230832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:05.231080 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:05.231180 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:05.231731 (+ 551us) server_negotiation.cc:299] Negotiation successful
0812 05:20:05.231816 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:20:05.233319 10399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: Bootstrap starting.
I20260812 05:20:05.234001 10399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:05.234326 10399 log.cc:824] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:05.235239 10399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: No bootstrap required, opened a new log
I20260812 05:20:05.237275 10399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.237488 10399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c09b2b6df694e169195cd995e71a092, State: Initialized, Role: FOLLOWER
I20260812 05:20:05.237609 10399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } }
I20260812 05:20:05.237983 10399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:05.238061 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c09b2b6df694e169195cd995e71a092"
I20260812 05:20:05.238161 10371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:05.239185 10371 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c4ffb4e9004c8d9a283f1e095d8ccb in term 1.
I20260812 05:20:05.239519 10370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [term 1 FOLLOWER]: Refusing update from remote peer e2c4ffb4e9004c8d9a283f1e095d8ccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:20:05.239915 10514 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:20:05.245679 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.246248 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:05.246309 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2c4ffb4e9004c8d9a283f1e095d8ccb. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.246367 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:05.246393 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.246438 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:20:05.247020 10540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:20:05.247073 10540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:20:05.247831 10542 mvcc.cc:205] Tried to move back new op lower bound from 7317553173417193472 to 7317553173275021312. Current Snapshot: MvccSnapshot[applied={T|T < 7317553173299376128}]
W20260812 05:20:05.273341 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.271614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.18.166.253:44657)
0812 05:20:05.271663 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.271665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.272623 (+ 958us) server_negotiation.cc:407] Connection header received
0812 05:20:05.272733 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.272734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.272746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.272779 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.273264 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38210: BlockingRecv error: recv got EOF from 127.0.0.1:38210 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 05:20:05.274363 10481 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.271978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.18.166.252:44693)
0812 05:20:05.272039 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.272042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.273816 (+ 1774us) server_negotiation.cc:407] Connection header received
0812 05:20:05.273915 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.273916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.273928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.273957 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.274301 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43092: BlockingRecv error: recv got EOF from 127.0.0.1:43092 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 05:20:05.275261 10323 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.272334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.18.166.254:33215)
0812 05:20:05.272379 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.272381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.274716 (+ 2335us) server_negotiation.cc:407] Connection header received
0812 05:20:05.274817 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.274818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.274830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.274861 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.275183 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52948: BlockingRecv error: recv got EOF from 127.0.0.1:52948 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:20:05.279203 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.278167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.18.166.253:44657)
0812 05:20:05.278208 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.278211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.278500 (+ 289us) server_negotiation.cc:407] Connection header received
0812 05:20:05.278627 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.278628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.278649 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.278680 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.279136 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38218: BlockingRecv error: recv got EOF from 127.0.0.1:38218 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:20:05.285948 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.281858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.18.166.253:44657)
0812 05:20:05.281895 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.281897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.285240 (+ 3343us) server_negotiation.cc:407] Connection header received
0812 05:20:05.285366 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.285367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.285380 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.285448 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.285888 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38228: BlockingRecv error: recv got EOF from 127.0.0.1:38228 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 05:20:05.290445 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.289519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.18.166.253:44657)
0812 05:20:05.289585 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.289587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.289856 (+ 269us) server_negotiation.cc:407] Connection header received
0812 05:20:05.289962 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.289963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.289974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.290003 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.290387 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38240: BlockingRecv error: recv got EOF from 127.0.0.1:38240 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 05:20:05.297000 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.295975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.18.166.253:44657)
0812 05:20:05.296021 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.296023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.296367 (+ 344us) server_negotiation.cc:407] Connection header received
0812 05:20:05.296486 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.296487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.296500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.296533 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.296939 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38242: BlockingRecv error: recv got EOF from 127.0.0.1:38242 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:20:05.307789 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.306750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.18.166.253:44657)
0812 05:20:05.306801 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.306803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.307138 (+ 335us) server_negotiation.cc:407] Connection header received
0812 05:20:05.307257 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.307258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.307271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.307303 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.307725 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38250: BlockingRecv error: recv got EOF from 127.0.0.1:38250 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 05:20:05.326422 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.325347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.18.166.253:44657)
0812 05:20:05.325397 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.325399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.325761 (+ 362us) server_negotiation.cc:407] Connection header received
0812 05:20:05.325872 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.325873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.325887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.325920 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.326357 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38264: BlockingRecv error: recv got EOF from 127.0.0.1:38264 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 05:20:05.360898 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.359952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.18.166.253:44657)
0812 05:20:05.359998 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.360000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.360300 (+ 300us) server_negotiation.cc:407] Connection header received
0812 05:20:05.360402 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.360403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.360415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.360444 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.360842 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38270: BlockingRecv error: recv got EOF from 127.0.0.1:38270 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 05:20:05.427325 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.426527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.18.166.253:44657)
0812 05:20:05.426591 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.426593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.426721 (+ 128us) server_negotiation.cc:407] Connection header received
0812 05:20:05.426769 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.426770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.426788 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.426870 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.427232 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38282: BlockingRecv error: recv got EOF from 127.0.0.1:38282 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 05:20:05.557525 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.556661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.18.166.253:44657)
0812 05:20:05.556739 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.556741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.556857 (+ 116us) server_negotiation.cc:407] Connection header received
0812 05:20:05.556951 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.556952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.556971 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.557039 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.557452 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38294: BlockingRecv error: recv got EOF from 127.0.0.1:38294 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:20:05.739151 10604 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c4ffb4e9004c8d9a283f1e095d8ccb" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 33215 } } peers { permanent_uuid: "2c09b2b6df694e169195cd995e71a092" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 44657 } } peers { permanent_uuid: "857d9ccccac642fe9982e3bd92a5277d" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 44693 } } }
I20260812 05:20:05.739395 10604 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2c4ffb4e9004c8d9a283f1e095d8ccb [sys.catalog]: This master's current role is: LEADER
W20260812 05:20:05.815675 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:05.814897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.18.166.253:44657)
0812 05:20:05.814978 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:05.814980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:05.814991 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:20:05.815069 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:05.815070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:05.815080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:05.815170 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:05.815606 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38304: BlockingRecv error: recv got EOF from 127.0.0.1:38304 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:20:06.181216 10494 catalog_manager.cc:1259] Loaded cluster ID: 05d30bad004043a99dd367445b90a624
I20260812 05:20:06.181337 10494 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: loading cluster ID for follower catalog manager: success
I20260812 05:20:06.182183 10494 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: acquiring CA information for follower catalog manager: success
I20260812 05:20:06.182453 10494 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 857d9ccccac642fe9982e3bd92a5277d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:20:06.247530 10540 catalog_manager.cc:1259] Loaded cluster ID: 05d30bad004043a99dd367445b90a624
I20260812 05:20:06.247622 10540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: loading cluster ID for follower catalog manager: success
I20260812 05:20:06.248529 10540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: acquiring CA information for follower catalog manager: success
I20260812 05:20:06.248813 10540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c09b2b6df694e169195cd995e71a092: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:20:06.331951 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:06.329067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.18.166.253:44657)
0812 05:20:06.329249 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:06.329251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:06.329268 (+ 17us) server_negotiation.cc:407] Connection header received
0812 05:20:06.329355 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:06.329356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:06.329439 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:06.329551 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:06.329813 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.330320 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.331198 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.331400 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.331454 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:06.331608 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:06.331741 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:06.331807 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:06.331863 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:20:06.331879 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260812 05:20:06.336436 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:06.333916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.18.166.252:44693)
0812 05:20:06.334069 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:06.334070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:06.334099 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:20:06.334161 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:06.334161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:06.334200 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:06.334264 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:06.334339 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.334827 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.335520 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.335737 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.335804 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:06.336065 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:06.336190 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:06.336286 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:06.336333 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 05:20:06.336354 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260812 05:20:06.341240 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:06.338482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.18.166.252:44693)
0812 05:20:06.338520 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:06.338522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:06.338529 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:20:06.338689 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:06.338690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:06.338703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:06.338764 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:06.338950 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.339678 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.340425 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.340719 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.340787 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:06.340936 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:06.340969 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:06.341025 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:06.341111 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:20:06.341142 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:20:06.341466 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:06.338229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.18.166.253:44657)
0812 05:20:06.338325 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:06.338328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:06.338386 (+ 58us) server_negotiation.cc:407] Connection header received
0812 05:20:06.338555 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:06.338556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:06.338571 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:06.338623 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:06.338865 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.339646 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.340642 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.340931 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.340999 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:06.341151 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:06.341181 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:06.341209 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:06.341326 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 05:20:06.341359 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 05:20:06.341892 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:06.338232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51216 (local address 127.18.166.254:33215)
0812 05:20:06.338372 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:06.338374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:06.338472 (+ 98us) server_negotiation.cc:407] Connection header received
0812 05:20:06.338622 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:06.338623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:06.338687 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:06.338751 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:06.338959 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.339474 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.340554 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:06.340867 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:06.340943 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:06.341326 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:06.341533 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:06.341598 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:06.341776 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 05:20:06.341806 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260812 05:20:06.343550 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10246
I20260812 05:20:06.352753 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10327
I20260812 05:20:06.358765 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10406
2026-08-12T05:20:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2857 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:20:06 dist-test-slave-srgv krb5kdc[10633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:20:06 dist-test-slave-srgv krb5kdc[10633](info): set up 2 sockets
Aug 12 05:20:06 dist-test-slave-srgv krb5kdc[10633](info): commencing operation
krb5kdc: starting...
W20260812 05:20:07.435911 19099 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/test-user.keytab.
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:20:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.166.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
Entry for principal kudu/127.18.166.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab.
I20260812 05:20:07.564263 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36171
--webserver_interface=127.18.166.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:36171,127.18.166.253:39705,127.18.166.252:39207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:20:07.650465 10647 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:20:07.650765 10647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:07.650807 10647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:07.650851 10647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:07.652066 10647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:07.652148 10647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:07.652171 10647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:07.652192 10647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:07.652213 10647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:07.653782 10647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:36171,127.18.166.253:39705,127.18.166.252:39207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.254.keytab
--principal=kudu/127.18.166.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.254:36171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.166.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:07.654095 10647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:07.654351 10647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:07.659543 10653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:07.659538 10654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:07.659538 10656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:07.659727 10647 server_base.cc:1034] running on GCE node
I20260812 05:20:07.660044 10647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:07.660375 10647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:07.661520 10647 hybrid_clock.cc:648] HybridClock initialized: now 1786512007661502 us; error 31 us; skew 500 ppm
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:07.668694 10647 init.cc:377] Logged in from keytab as kudu/127.18.166.254@KRBTEST.COM (short username kudu)
I20260812 05:20:07.669214 10647 webserver.cc:460] Webserver started at http://127.18.166.254:33593/ using document root <none> and password file <none>
I20260812 05:20:07.669530 10647 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:07.669606 10647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:07.669788 10647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:07.670948 10647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data/instance:
uuid: "31705c4083cb48a6b882df78056c1b70"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "8f46cf6b6b1d39924972592e6e2eeb3e"
server_key_iv: "4b57da5a0e164dd3d44306fdc63583b8"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.671347 10647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal/instance:
uuid: "31705c4083cb48a6b882df78056c1b70"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "8f46cf6b6b1d39924972592e6e2eeb3e"
server_key_iv: "4b57da5a0e164dd3d44306fdc63583b8"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.672851 10647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:20:07.673812 10664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:07.674042 10647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:20:07.674157 10647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal
uuid: "31705c4083cb48a6b882df78056c1b70"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "8f46cf6b6b1d39924972592e6e2eeb3e"
server_key_iv: "4b57da5a0e164dd3d44306fdc63583b8"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.674240 10647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:07.681720 10647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:07.683926 10647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:07.684132 10647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:07.688606 10647 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.254:36171
I20260812 05:20:07.688642 10718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.254:36171 every 8 connection(s)
I20260812 05:20:07.689023 10647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/data/info.pb
I20260812 05:20:07.690028 10719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:07.690375 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10647
I20260812 05:20:07.690491 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-0/wal/instance
I20260812 05:20:07.690716 19099 external_mini_cluster.cc:1428] Setting key a56ce541413713b8635873044404c114
I20260812 05:20:07.691854 10719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.166.253@KRBTEST.COM; defaulting to no policy
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.253@KRBTEST.COM" created.
I20260812 05:20:07.703934 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.692851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.18.166.254:36171)
0812 05:20:07.692989 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:07.692993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:07.693033 (+ 40us) server_negotiation.cc:407] Connection header received
0812 05:20:07.693904 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:07.693906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:07.693934 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:07.694006 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:07.694899 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.695865 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.697365 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.697762 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.697852 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.700476 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:07.700484 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:07.700484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:07.700491 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:07.702182 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.702573 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.702576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.702577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.702666 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.702916 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.702917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.702918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.703263 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:07.703352 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:07.703621 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 05:20:07.703726 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260812 05:20:07.704098 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.692679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36171 (local address 127.0.0.1:43258)
0812 05:20:07.692869 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.692921 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:07.693726 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:07.694057 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:07.694060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:07.694485 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:07.694735 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.694742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.696017 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.696018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.697249 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.697253 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.697864 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.697865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.697954 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.698545 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:07.698556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:07.700366 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:07.702276 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.702284 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.702285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.702493 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.702738 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.702739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.702739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.702842 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.703359 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:07.703368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:07.703458 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 05:20:07.703583 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 05:20:07.703855 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":69,"thread_start_us":84,"threads_started":1}
I20260812 05:20:07.704850 10719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:07.705261 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.705130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59434)
0812 05:20:07.705186 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.705242 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 05:20:07.705457 10668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
I20260812 05:20:07.705858 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.705793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59436)
0812 05:20:07.705819 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.705842 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 05:20:07.705983 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:39705: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab.
Entry for principal kudu/127.18.166.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab.
I20260812 05:20:07.720538 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:39705
--webserver_interface=127.18.166.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab
--principal=kudu/127.18.166.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:36171,127.18.166.253:39705,127.18.166.252:39207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:20:07.755241 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } attempt: 1
I20260812 05:20:07.755779 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.755628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59442)
0812 05:20:07.755696 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.755745 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:20:07.756161 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.756064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59444)
0812 05:20:07.756111 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.756141 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 05:20:07.756314 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:39705: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
W20260812 05:20:07.804467 10727 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:20:07.804705 10727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:07.804723 10727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:07.804746 10727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:07.806046 10727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:07.806095 10727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:07.806109 10727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:07.806123 10727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:07.806169 10727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:07.807708 10727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:36171,127.18.166.253:39705,127.18.166.252:39207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.253.keytab
--principal=kudu/127.18.166.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.253:39705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.166.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:07.807992 10727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:07.808238 10727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:07.813237 10732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:07.813297 10733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:07.813232 10735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:07.813558 10727 server_base.cc:1034] running on GCE node
I20260812 05:20:07.813817 10727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:07.814127 10727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:07.815276 10727 hybrid_clock.cc:648] HybridClock initialized: now 1786512007815259 us; error 45 us; skew 500 ppm
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:07.817900 10727 init.cc:377] Logged in from keytab as kudu/127.18.166.253@KRBTEST.COM (short username kudu)
I20260812 05:20:07.818398 10727 webserver.cc:460] Webserver started at http://127.18.166.253:37635/ using document root <none> and password file <none>
I20260812 05:20:07.818626 10727 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:07.818671 10727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:07.818846 10727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:07.819940 10727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data/instance:
uuid: "764b69c171ed47bd9871ad18bc7e520e"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "d84d5b4d98b67cf3f8ef76c5defec2de"
server_key_iv: "ce956e9a380566b064bb2a5233474822"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.820317 10727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal/instance:
uuid: "764b69c171ed47bd9871ad18bc7e520e"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "d84d5b4d98b67cf3f8ef76c5defec2de"
server_key_iv: "ce956e9a380566b064bb2a5233474822"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.821713 10727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:20:07.822427 10743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:07.822623 10727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:20:07.822716 10727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal
uuid: "764b69c171ed47bd9871ad18bc7e520e"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "d84d5b4d98b67cf3f8ef76c5defec2de"
server_key_iv: "ce956e9a380566b064bb2a5233474822"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.822798 10727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:07.824506 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } attempt: 2
I20260812 05:20:07.824965 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.824811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59456)
0812 05:20:07.824878 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.824924 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:20:07.825354 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.825252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59470)
0812 05:20:07.825307 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.825334 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 05:20:07.825539 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.253:39705: Network error: Client connection negotiation failed: client connection to 127.18.166.253:39705: connect: Connection refused (error 111)
I20260812 05:20:07.832824 10727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:07.834847 10727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:07.835018 10727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:07.839154 10727 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.253:39705
I20260812 05:20:07.839188 10800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.253:39705 every 8 connection(s)
I20260812 05:20:07.839541 10727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/data/info.pb
I20260812 05:20:07.840416 10801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:07.842024 10801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:07.845857 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10727
I20260812 05:20:07.845979 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-1/wal/instance
I20260812 05:20:07.846168 19099 external_mini_cluster.cc:1428] Setting key f2677167b29c56d9d2c55ceff4d4e8f4
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:20:07.852133 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.842678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36171 (local address 127.0.0.1:43260)
0812 05:20:07.842837 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.842879 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:07.843338 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:07.843598 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:07.843601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:07.843848 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:07.844033 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.844037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.844679 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.844680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.845525 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.845528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.845931 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.845933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.846092 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.846654 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:07.846664 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:07.848563 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:07.850642 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.850653 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.850656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.850918 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.851171 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.851171 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.851172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.851282 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.851611 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:07.851617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:07.851739 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 05:20:07.851828 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:20:07.851954 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260812 05:20:07.852133 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.842707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.18.166.254:36171)
0812 05:20:07.842780 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:07.842782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:07.842971 (+ 189us) server_negotiation.cc:407] Connection header received
0812 05:20:07.843445 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:07.843446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:07.843458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:07.843532 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:07.844126 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.844598 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.845613 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.845831 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.845893 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.848677 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:07.848679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:07.848679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:07.848686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:07.850513 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.851007 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.851009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.851009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.851091 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.851357 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.851359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.851359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.851527 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:07.851613 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:07.851879 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 05:20:07.851980 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:20:07.852825 10801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.166.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.166.252@KRBTEST.COM" created.
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:07.862414 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.853191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.18.166.253:39705)
0812 05:20:07.853358 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:07.853363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:07.853382 (+ 19us) server_negotiation.cc:407] Connection header received
0812 05:20:07.853447 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:07.853450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:07.853463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:07.853539 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:07.853884 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.854574 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.855672 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.856074 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.856142 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.859064 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:07.859075 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:07.859075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:07.859083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:07.861045 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.861523 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.861526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.861526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.861581 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.861801 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.861801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.861802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.861944 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:07.862007 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:07.862175 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 05:20:07.862325 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":138,"thread_start_us":111,"threads_started":1}
I20260812 05:20:07.862424 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.853120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59472)
0812 05:20:07.853182 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.853195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:07.853240 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:07.853560 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:07.853561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:07.853734 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:07.853800 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.853803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.854669 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.854671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.855533 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.855536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.856184 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.856185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.856249 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.856732 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:07.856736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:07.858971 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:07.861159 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.861161 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.861162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.861401 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.861652 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.861653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.861654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.861737 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.862012 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:07.862013 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:07.862064 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:20:07.862139 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:20:07.862259 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:20:07.863497 10801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:07.863935 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.863785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35758)
0812 05:20:07.863842 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.863902 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 05:20:07.864218 10746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
I20260812 05:20:07.864734 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.864605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35774)
0812 05:20:07.864658 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.864701 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:20:07.864988 10801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39207: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.166.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab.
Entry for principal kudu/127.18.166.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab.
I20260812 05:20:07.876150 19099 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
/tmp/dist-test-taskjJH1Yp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:39207
--webserver_interface=127.18.166.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab
--principal=kudu/127.18.166.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.166.212:46695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.166.254:36171,127.18.166.253:39705,127.18.166.252:39207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:20:07.914297 10801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } attempt: 1
I20260812 05:20:07.914939 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.914772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35786)
0812 05:20:07.914845 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.914900 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 05:20:07.915315 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.915220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35794)
0812 05:20:07.915265 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.915294 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 05:20:07.915444 10801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39207: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
I20260812 05:20:07.916678 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } attempt: 3
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:07.924603 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.916998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59482)
0812 05:20:07.917050 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.917061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:07.917115 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:07.917291 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:07.917292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:07.917494 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:07.917563 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.917565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.918243 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.918245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.918950 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:07.918953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.919567 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.919569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:07.919685 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.920207 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:07.920214 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:07.921734 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:07.923491 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.923492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.923492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.923679 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.923898 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:07.923899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:07.923899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:07.923968 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:07.924249 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:07.924251 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:07.924301 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 05:20:07.924391 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:20:07.924500 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 05:20:07.924602 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.917054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.18.166.253:39705)
0812 05:20:07.917126 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:07.917128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:07.917156 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:20:07.917182 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:07.917183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:07.917195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:07.917282 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:07.917652 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.918155 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.919044 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:07.919342 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:07.919568 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:07.921841 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:07.921843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:07.921844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:07.921851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:07.923357 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.923771 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.923772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.923772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.923832 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:07.924040 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:07.924041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:07.924041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:07.924173 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:07.924258 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:07.924417 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 05:20:07.924501 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:20:07.925016 10719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:07.925384 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.925277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35804)
0812 05:20:07.925330 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.925358 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:20:07.925820 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.925715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35814)
0812 05:20:07.925765 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.925803 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 05:20:07.925927 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39207: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
I20260812 05:20:07.957077 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } attempt: 1
I20260812 05:20:07.957635 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.957500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35826)
0812 05:20:07.957566 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.957608 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:20:07.957960 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.957902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35832)
0812 05:20:07.957924 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.957946 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 05:20:07.958069 10719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39207: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
W20260812 05:20:07.964512 10808 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:20:07.964735 10808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:20:07.964753 10808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:20:07.964774 10808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:20:07.966107 10808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:20:07.966154 10808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:20:07.966168 10808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:20:07.966187 10808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:20:07.966239 10808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:20:07.967790 10808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.166.212:46695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.166.254:36171,127.18.166.253:39705,127.18.166.252:39207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/krb5kdc/kudu_127.18.166.252.keytab
--principal=kudu/127.18.166.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.166.252:39207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.166.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:20:07.968075 10808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:20:07.968338 10808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:20:07.973196 10814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:07.973263 10816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:20:07.973273 10813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:20:07.973558 10808 server_base.cc:1034] running on GCE node
I20260812 05:20:07.973830 10808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:20:07.974128 10808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:20:07.975304 10808 hybrid_clock.cc:648] HybridClock initialized: now 1786512007975289 us; error 29 us; skew 500 ppm
Aug 12 05:20:07 dist-test-slave-srgv krb5kdc[10633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:20:07.978103 10808 init.cc:377] Logged in from keytab as kudu/127.18.166.252@KRBTEST.COM (short username kudu)
I20260812 05:20:07.978582 10808 webserver.cc:460] Webserver started at http://127.18.166.252:32853/ using document root <none> and password file <none>
I20260812 05:20:07.978835 10808 fs_manager.cc:359] Metadata directory not provided
I20260812 05:20:07.978907 10808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:20:07.979063 10808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:20:07.980165 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data/instance:
uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "2450d734f532b5cffa89fd2c82222529"
server_key_iv: "dbb6eaf573c4b3b7733167db922c6617"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.980563 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal/instance:
uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "2450d734f532b5cffa89fd2c82222529"
server_key_iv: "dbb6eaf573c4b3b7733167db922c6617"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.982111 10808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:20:07.982940 10823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:20:07.983132 10808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:20:07.983256 10808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data,/tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal
uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5"
format_stamp: "Formatted at 2026-08-12 05:20:07 on dist-test-slave-srgv"
server_key: "2450d734f532b5cffa89fd2c82222529"
server_key_iv: "dbb6eaf573c4b3b7733167db922c6617"
server_key_version: "encryptionkey@0"
I20260812 05:20:07.983336 10808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:20:07.983630 10801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } attempt: 2
I20260812 05:20:07.984117 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.983954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35842)
0812 05:20:07.984018 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.984072 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:20:07.984601 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:07.984453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35848)
0812 05:20:07.984539 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:07.984575 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 05:20:07.984743 10801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.166.252:39207: Network error: Client connection negotiation failed: client connection to 127.18.166.252:39207: connect: Connection refused (error 111)
I20260812 05:20:08.007475 10808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:20:08.010076 10808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:20:08.010257 10808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:20:08.014555 10808 rpc_server.cc:307] RPC server started. Bound to: 127.18.166.252:39207
I20260812 05:20:08.014591 10880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.166.252:39207 every 8 connection(s)
I20260812 05:20:08.014983 10808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/data/info.pb
I20260812 05:20:08.015969 10881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:20:08.017644 10881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:20:08.022274 19099 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu as pid 10808
I20260812 05:20:08.022416 19099 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskjJH1Yp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786511731160932-19099-0/minicluster-data/master-2/wal/instance
I20260812 05:20:08.022660 19099 external_mini_cluster.cc:1428] Setting key 0e7afd1edf189fe5d0a3d706a8080f03
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
I20260812 05:20:08.027375 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.018421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.18.166.254:36171)
0812 05:20:08.018527 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.018530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.018733 (+ 203us) server_negotiation.cc:407] Connection header received
0812 05:20:08.019214 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.019215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.019234 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.019296 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.019935 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.020419 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.021350 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.021607 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.021663 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.024172 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.024173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.024174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.024180 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.025737 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.026225 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.026226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.026227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.026302 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.026589 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.026590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.026590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.026743 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.026806 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.027186 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.027307 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 05:20:08.027591 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.018366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.254:36171 (local address 127.0.0.1:43270)
0812 05:20:08.018571 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:08.018609 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:08.019084 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:08.019373 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:08.019376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:08.019640 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:08.019846 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.019851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.020500 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.020501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.021271 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.021273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.021684 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.021685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.021799 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.022424 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:08.022432 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:08.024011 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:08.025856 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.025863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.025865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.026123 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.026383 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.026384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.026385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.026511 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.026822 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:08.026829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:08.026994 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 05:20:08.027124 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 05:20:08.027300 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260812 05:20:08.028229 10881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:08.032297 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.023083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.18.166.252:39207)
0812 05:20:08.023284 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.023289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.023318 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:20:08.023384 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.023387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.023415 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.023490 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.024177 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.024846 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.025681 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.026074 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.026169 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.029087 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.029094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.029095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.029101 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.030892 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.031330 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.031332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.031332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.031387 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.031647 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.031649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.031649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.031861 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.032053 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.032099 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.032179 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260812 05:20:08.033198 10719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } attempt: 2
I20260812 05:20:08.036103 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.028491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.253:39705 (local address 127.0.0.1:59488)
0812 05:20:08.028540 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:08.028551 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:08.028589 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:08.028731 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:08.028731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:08.028844 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:08.028888 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.028889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.029631 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.029632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.030128 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.030130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.030534 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.030535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.030605 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.031027 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:08.031032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:08.032580 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:08.035023 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.035024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.035025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.035200 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.035436 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.035437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.035438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.035501 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.035772 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:08.035773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:08.035817 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 05:20:08.035899 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 05:20:08.036001 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 05:20:08.036105 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.028574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59488 (local address 127.18.166.253:39705)
0812 05:20:08.028622 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.028624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.028632 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:20:08.028658 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.028659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.028670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.028731 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.028967 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.029553 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.030232 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.030456 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.030536 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.032667 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.032669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.032669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.032675 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.034914 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.035295 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.035296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.035296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.035362 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.035564 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.035565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.035565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.035685 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.035780 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.035940 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.036018 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:20:08.037726 10881 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.254@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.254@KRBTEST.COM
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.252@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:20:08.042420 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.033471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35858)
0812 05:20:08.033511 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:08.033523 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:08.033560 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:08.033780 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:08.033781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:08.033964 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:08.034014 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.034016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.034940 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.034941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.036142 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.036143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.036608 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.036609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.036672 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.037044 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:08.037050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:08.039150 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:08.041055 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.041056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.041056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.041506 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.041748 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.041749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.041750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.041819 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.042105 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:08.042106 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:08.042150 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 05:20:08.042207 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:20:08.042322 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 05:20:08.042419 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.033491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.18.166.252:39207)
0812 05:20:08.033650 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.033652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.033659 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:20:08.033677 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.033678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.033687 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.033750 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.034111 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.034851 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.036228 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.036495 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.036596 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.039238 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.039238 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.039239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.039244 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.040955 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.041600 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.041601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.041602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.041657 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.041889 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.041889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.041890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.042022 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.042114 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.042253 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.042322 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 05:20:08.043085 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.033222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.18.166.254:36171)
0812 05:20:08.033273 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.033275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.033356 (+ 81us) server_negotiation.cc:407] Connection header received
0812 05:20:08.033531 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.033532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.033541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.033602 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.034299 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.034958 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.035970 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.036261 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.036298 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.039673 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.039674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.039674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.039681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.041554 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.041966 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.041967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.041967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.042013 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.042239 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.042240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.042240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.042377 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.042625 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.042900 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.043009 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:20:08.044451 10719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70: Bootstrap starting.
I20260812 05:20:08.045246 10719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:08.045598 10719 log.cc:824] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:08.045897 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.038057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.18.166.252:39207)
0812 05:20:08.038247 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.038250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.038258 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:20:08.038278 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.038279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.038291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.038376 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.038594 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.039350 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.040292 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.040605 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.040873 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.042685 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.042686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.042687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.042693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.044539 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.044935 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.044936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.044937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.045000 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.045217 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.045218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.045219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.045456 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.045524 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.045657 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.045733 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260812 05:20:08.045902 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.038007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35870)
0812 05:20:08.038057 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:08.038069 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:08.038107 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:08.038378 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:08.038378 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:08.038489 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:08.038518 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.038519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.039441 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.039442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.040190 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.040193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.040688 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.040688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.040741 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.041109 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:08.041114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:08.042599 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:08.044616 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.044617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.044617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.044851 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.045084 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.045085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.045085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.045158 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.045514 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:08.045515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:08.045554 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 05:20:08.045622 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 05:20:08.045799 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":17}
I20260812 05:20:08.046411 10719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70: No bootstrap required, opened a new log
I20260812 05:20:08.047685 10881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: Bootstrap starting.
I20260812 05:20:08.048296 10881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:08.048588 10881 log.cc:824] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:08.048599 10719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.048763 10719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31705c4083cb48a6b882df78056c1b70, State: Initialized, Role: FOLLOWER
I20260812 05:20:08.048862 10719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.049156 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.049255 10719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:08.049265 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.166.253@KRBTEST.COM
I20260812 05:20:08.050674 10881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: No bootstrap required, opened a new log
W20260812 05:20:08.052417 10901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:20:08.052466 10901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:20:08.052783 10881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.052937 10881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 467a9bcc4ba343ec8a5ee3f4a6f487c5, State: Initialized, Role: FOLLOWER
I20260812 05:20:08.053030 10881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.053349 10881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:08.053344 10904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.053516 10904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:08.053848 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.044350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59492 (local address 127.18.166.253:39705)
0812 05:20:08.044392 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.044394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.044403 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:20:08.044430 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.044431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.044440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.044506 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.045176 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.045917 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.046853 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.047121 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.047197 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.050054 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.050056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.050056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.050064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.051889 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.052315 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.052316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.052316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.052382 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.052648 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.052650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.052650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.052790 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.052942 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.053647 (+ 705us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.053741 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:20:08.052243 10647 master_runner.cc:186] Error getting master registration for 127.18.166.253:39705: OK, instance_id { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" instance_seqno: 1786512007838008 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:20:08.054991 10647 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 05:20:08.055187 10915 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:20:08.055243 10915 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:20:08.056241 19099 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:20:08.057030 10693 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "31705c4083cb48a6b882df78056c1b70"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43272
I20260812 05:20:08.057142 10693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 05:20:08.057189 10693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:08.058205 10693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.058424 10693 leader_election.cc:288] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [CANDIDATE]: Term 1 election: Requested vote from peers 764b69c171ed47bd9871ad18bc7e520e (127.18.166.253:39705), 467a9bcc4ba343ec8a5ee3f4a6f487c5 (127.18.166.252:39207)
I20260812 05:20:08.058669 10852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31705c4083cb48a6b882df78056c1b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5"
I20260812 05:20:08.058768 10852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:08.057188 10808 master_runner.cc:186] Error getting master registration for 127.18.166.253:39705: OK, instance_id { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" instance_seqno: 1786512007838008 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:20:08.059332 10808 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:20:08.059636 10852 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31705c4083cb48a6b882df78056c1b70 in term 1.
I20260812 05:20:08.059830 10668 leader_election.cc:302] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31705c4083cb48a6b882df78056c1b70, 467a9bcc4ba343ec8a5ee3f4a6f487c5; no voters:
I20260812 05:20:08.059952 10889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:20:08.060117 10889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [term 1 LEADER]: Becoming Leader. State: Replica: 31705c4083cb48a6b882df78056c1b70, State: Running, Role: LEADER
I20260812 05:20:08.060441 10889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.060899 10921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31705c4083cb48a6b882df78056c1b70. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.060999 10921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:08.061204 10923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:20:08.061424 10923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:20:08.062914 10852 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [term 1 FOLLOWER]: Refusing update from remote peer 31705c4083cb48a6b882df78056c1b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:20:08.063274 10889 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:20:08.064527 10904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31705c4083cb48a6b882df78056c1b70. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.064649 10904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:08.076256 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.065711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.18.166.253:39705)
0812 05:20:08.065764 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.065766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.065993 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:20:08.066108 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.066109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.066119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.066149 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.067210 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.067983 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.069520 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.069841 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.069884 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.072445 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.072447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.072448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.072457 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.074269 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.074935 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.074937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.074937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.075054 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.075441 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.075442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.075443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.075605 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.075684 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.076046 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.076135 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:20:08.076259 10801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } attempt: 3
I20260812 05:20:08.077755 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.077863 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:08.078045 10904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.078173 10904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:08.078469 10925 mvcc.cc:205] Tried to move back new op lower bound from 7317553185023401984 to 7317553185016365056. Current Snapshot: MvccSnapshot[applied={T|T < 7317553185023401984}]
Aug 12 05:20:08 dist-test-slave-srgv krb5kdc[10633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512007, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.166.253@KRBTEST.COM for kudu/127.18.166.252@KRBTEST.COM
I20260812 05:20:08.081842 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.066033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.18.166.252:39207)
0812 05:20:08.066070 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.066073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.066489 (+ 416us) server_negotiation.cc:407] Connection header received
0812 05:20:08.066673 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.066674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.066686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.066719 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.067845 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.068564 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.070071 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.070806 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.070858 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.073807 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.073808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.073809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.073817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.076181 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.080854 (+ 4673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.080855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.080855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.080912 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.081170 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.081171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.081171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.081302 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.081451 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.081639 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.081733 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:20:08.084347 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.066223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.18.166.254:36171)
0812 05:20:08.066298 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.066300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.073589 (+ 7289us) server_negotiation.cc:407] Connection header received
0812 05:20:08.073697 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.073698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.073711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.073762 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.074500 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.075304 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.076324 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.076584 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.076628 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.078729 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.078730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.078731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.078752 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.080350 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.082254 (+ 1904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.082256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.082256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.082325 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.082916 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.082918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.082918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.083062 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.083174 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.084150 (+ 976us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.084244 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:20:08.085448 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.076531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.166.252:39207 (local address 127.0.0.1:35888)
0812 05:20:08.076611 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 05:20:08.076621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:20:08.076655 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:20:08.076915 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:20:08.076916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:20:08.077074 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:20:08.077115 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.077117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.078375 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.078376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.079173 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:20:08.079176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.079692 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.079693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:20:08.079757 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.080108 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:20:08.080113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:20:08.081660 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:20:08.083922 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.083923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.083923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.084170 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.084790 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:20:08.084790 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:20:08.084791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:20:08.084850 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:20:08.085106 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:20:08.085107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:20:08.085164 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 05:20:08.085235 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:20:08.085333 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 05:20:08.085446 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:08.076728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.18.166.252:39207)
0812 05:20:08.076768 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.076770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.076777 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:20:08.076797 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.076798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.076808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.076857 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.077225 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.078277 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.079280 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:08.079581 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:08.079641 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:08.081769 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:20:08.081770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:20:08.081770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:20:08.081778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:20:08.083848 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.084650 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.084651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.084651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.084710 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:20:08.084914 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:20:08.084914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:20:08.084915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:20:08.085036 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:20:08.085099 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:08.085262 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:20:08.085335 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:20:08.087332 10801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: Bootstrap starting.
I20260812 05:20:08.087991 10801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: Neither blocks nor log segments found. Creating new log.
I20260812 05:20:08.088292 10801 log.cc:824] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: Log is configured to *not* fsync() on all Append() calls
I20260812 05:20:08.089151 10801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: No bootstrap required, opened a new log
I20260812 05:20:08.090268 10923 catalog_manager.cc:1347] Generated new cluster ID: 363907acb8574d65b5289c8bb568e947
I20260812 05:20:08.090327 10923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:20:08.091270 10801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.091416 10801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 764b69c171ed47bd9871ad18bc7e520e, State: Initialized, Role: FOLLOWER
I20260812 05:20:08.091500 10801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } }
I20260812 05:20:08.091900 10772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31705c4083cb48a6b882df78056c1b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "764b69c171ed47bd9871ad18bc7e520e"
I20260812 05:20:08.092223 10771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:20:08.093353 10771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [term 1 FOLLOWER]: Refusing update from remote peer 31705c4083cb48a6b882df78056c1b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:20:08.093766 10921 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:20:08.091867 10801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:20:08.096365 10772 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31705c4083cb48a6b882df78056c1b70 in term 1.
I20260812 05:20:08.096521 10923 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:20:08.097247 10923 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:20:08.091806 10933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.098328 10945 mvcc.cc:205] Tried to move back new op lower bound from 7317553185023401984 to 7317553185016365056. Current Snapshot: MvccSnapshot[applied={T|T < 7317553185023401984}]
I20260812 05:20:08.097872 10933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:08.098560 10933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31705c4083cb48a6b882df78056c1b70. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.098767 10933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:08.098831 10933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.098878 10933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:20:08.106570 10923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70: Generated new TSK 0
I20260812 05:20:08.106973 10923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:20:08.114773 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31705c4083cb48a6b882df78056c1b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31705c4083cb48a6b882df78056c1b70" member_type: VOTER last_known_addr { host: "127.18.166.254" port: 36171 } } peers { permanent_uuid: "764b69c171ed47bd9871ad18bc7e520e" member_type: VOTER last_known_addr { host: "127.18.166.253" port: 39705 } } peers { permanent_uuid: "467a9bcc4ba343ec8a5ee3f4a6f487c5" member_type: VOTER last_known_addr { host: "127.18.166.252" port: 39207 } } }
I20260812 05:20:08.114887 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31705c4083cb48a6b882df78056c1b70 [sys.catalog]: This master's current role is: LEADER
I20260812 05:20:08.115119 10951 catalog_manager.cc:1259] Loaded cluster ID: 363907acb8574d65b5289c8bb568e947
I20260812 05:20:08.115204 10951 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: loading cluster ID for follower catalog manager: success
W20260812 05:20:08.115373 10951 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260812 05:20:08.115477 10951 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260812 05:20:08.143003 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.141175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59506 (local address 127.18.166.253:39705)
0812 05:20:08.141237 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.141239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.142296 (+ 1057us) server_negotiation.cc:407] Connection header received
0812 05:20:08.142409 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.142411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.142422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.142454 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.142931 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59506: BlockingRecv error: recv got EOF from 127.0.0.1:59506 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 05:20:08.143805 10885 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.141625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.18.166.252:39207)
0812 05:20:08.141668 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.141670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.143286 (+ 1616us) server_negotiation.cc:407] Connection header received
0812 05:20:08.143386 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.143387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.143398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.143428 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.143740 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35900: BlockingRecv error: recv got EOF from 127.0.0.1:35900 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 05:20:08.144575 10723 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.141983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.18.166.254:36171)
0812 05:20:08.142024 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.142026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.144034 (+ 2008us) server_negotiation.cc:407] Connection header received
0812 05:20:08.144158 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.144159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.144171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.144202 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.144513 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43302: BlockingRecv error: recv got EOF from 127.0.0.1:43302 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:20:08.147039 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.145857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.18.166.253:39705)
0812 05:20:08.145892 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.145894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.146488 (+ 594us) server_negotiation.cc:407] Connection header received
0812 05:20:08.146622 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.146623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.146643 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.146673 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.146988 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59508: BlockingRecv error: recv got EOF from 127.0.0.1:59508 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 05:20:08.150413 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.149327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.18.166.253:39705)
0812 05:20:08.149360 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.149362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.149636 (+ 274us) server_negotiation.cc:407] Connection header received
0812 05:20:08.149783 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.149784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.149794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.149824 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.150363 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59512: BlockingRecv error: recv got EOF from 127.0.0.1:59512 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 05:20:08.154295 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.153480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.18.166.253:39705)
0812 05:20:08.153516 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.153518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.153807 (+ 289us) server_negotiation.cc:407] Connection header received
0812 05:20:08.153896 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.153897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.153907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.153935 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.154249 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59522: BlockingRecv error: recv got EOF from 127.0.0.1:59522 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 05:20:08.160180 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.159370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.18.166.253:39705)
0812 05:20:08.159407 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.159409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.159657 (+ 248us) server_negotiation.cc:407] Connection header received
0812 05:20:08.159758 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.159759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.159770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.159801 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.160128 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59524: BlockingRecv error: recv got EOF from 127.0.0.1:59524 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:20:08.170447 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.169487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.18.166.253:39705)
0812 05:20:08.169544 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.169545 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.169841 (+ 296us) server_negotiation.cc:407] Connection header received
0812 05:20:08.169949 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.169950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.169962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.169992 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.170375 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59528: BlockingRecv error: recv got EOF from 127.0.0.1:59528 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 05:20:08.188774 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.187778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59532 (local address 127.18.166.253:39705)
0812 05:20:08.187835 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.187837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.188143 (+ 306us) server_negotiation.cc:407] Connection header received
0812 05:20:08.188272 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.188273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.188285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.188316 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.188711 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59532: BlockingRecv error: recv got EOF from 127.0.0.1:59532 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 05:20:08.223228 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.222238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.18.166.253:39705)
0812 05:20:08.222291 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.222293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.222587 (+ 294us) server_negotiation.cc:407] Connection header received
0812 05:20:08.222689 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.222690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.222702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.222738 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.223149 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59546: BlockingRecv error: recv got EOF from 127.0.0.1:59546 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 05:20:08.289628 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.288787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.18.166.253:39705)
0812 05:20:08.288869 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.288872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.288977 (+ 105us) server_negotiation.cc:407] Connection header received
0812 05:20:08.289090 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.289092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.289104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.289160 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.289556 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59548: BlockingRecv error: recv got EOF from 127.0.0.1:59548 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 05:20:08.419595 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.418887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.18.166.253:39705)
0812 05:20:08.418986 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.418988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.419000 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:20:08.419085 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.419086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.419097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.419150 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.419527 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59564: BlockingRecv error: recv got EOF from 127.0.0.1:59564 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 05:20:08.677680 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:20:08.676914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59576 (local address 127.18.166.253:39705)
0812 05:20:08.676999 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:08.677001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:08.677031 (+ 30us) server_negotiation.cc:407] Connection header received
0812 05:20:08.677111 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:08.677112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:08.677122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:08.677200 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:20:08.677614 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59576: BlockingRecv error: recv got EOF from 127.0.0.1:59576 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 05:20:09.055675 10915 catalog_manager.cc:1259] Loaded cluster ID: 363907acb8574d65b5289c8bb568e947
I20260812 05:20:09.055763 10915 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: loading cluster ID for follower catalog manager: success
I20260812 05:20:09.056535 10915 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: acquiring CA information for follower catalog manager: success
I20260812 05:20:09.056803 10915 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 467a9bcc4ba343ec8a5ee3f4a6f487c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:20:09.116647 10951 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: acquiring CA information for follower catalog manager: success
I20260812 05:20:09.117071 10951 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 764b69c171ed47bd9871ad18bc7e520e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:20:09.193746 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:09.190921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.18.166.253:39705)
0812 05:20:09.191039 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:09.191041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:09.191072 (+ 31us) server_negotiation.cc:407] Connection header received
0812 05:20:09.191196 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:09.191196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:09.191252 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:09.191342 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:09.191494 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.192006 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.192836 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.193042 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.193110 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:09.193338 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:09.193477 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:09.193544 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:09.193599 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 05:20:09.193621 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260812 05:20:09.198658 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:09.196014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.18.166.252:39207)
0812 05:20:09.196156 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:09.196158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:09.196167 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:20:09.196219 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:09.196220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:09.196260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:09.196335 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:09.196495 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.197006 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.197888 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.198095 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.198175 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:09.198342 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:09.198461 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:09.198523 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:09.198559 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 05:20:09.198580 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260812 05:20:09.203385 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:09.200396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59592 (local address 127.18.166.253:39705)
0812 05:20:09.200478 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:09.200481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:09.200625 (+ 144us) server_negotiation.cc:407] Connection header received
0812 05:20:09.200980 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:09.200981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:09.200995 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:09.201045 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:09.201259 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.201835 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.202616 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.202839 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.202923 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:09.203095 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:09.203132 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:09.203196 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:09.203261 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 05:20:09.203284 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:20:09.203951 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:09.200745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.18.166.252:39207)
0812 05:20:09.200820 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:09.200821 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:09.200872 (+ 51us) server_negotiation.cc:407] Connection header received
0812 05:20:09.200941 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:09.200942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:09.200952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:09.201009 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:09.201120 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.201896 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.203107 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.203423 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.203466 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:09.203683 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:09.203715 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:09.203750 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:09.203851 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 05:20:09.203874 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:20:09.204295 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:20:09.200397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.18.166.254:36171)
0812 05:20:09.200522 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 05:20:09.200525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:20:09.200635 (+ 110us) server_negotiation.cc:407] Connection header received
0812 05:20:09.200756 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:20:09.200758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:20:09.200819 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:20:09.200911 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:20:09.201070 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.201900 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.203107 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:20:09.203433 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:20:09.203469 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:20:09.203883 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:20:09.204008 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:20:09.204090 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:20:09.204186 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 05:20:09.204214 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260812 05:20:09.206296 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10647
I20260812 05:20:09.212990 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10727
I20260812 05:20:09.219363 19099 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskjJH1Yp/build/release/bin/kudu with pid 10808
2026-08-12T05:20:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2859 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5716 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278054 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST